Structured Diversity Control: A Dual-Level Framework for Group-Aware Multi-Agent Coordination
Shuocun Yang, Huawen Hu, Xuan Liu, Yincheng Yao, Enze Shi, Shu Zhang
AI summary
Problem
Existing MARL diversity control methods treat agents as a single population, ignoring group structures and failing to simultaneously enforce tight intra-group cooperation and distinct inter-group specialization.
Approach
SDC decomposes system-wide diversity into intra-group and inter-group components, using a tunable Diversity Structure Factor to scale policy deviations and guide collective strategy without modifying reward functions.
Key results
- Precise regulation of structured system diversity to match target levels
- Up to 47.1% reward increase in multi-target pursuit tasks
- 12.82% reduction in episode lengths in complex neutralization scenarios
- Successful emergence of group-aware division of labor outperforming DiCo baseline
Why it matters
Enables MARL researchers and practitioners to explicitly engineer cooperative strategies in grouped multi-agent systems, improving scalability and task performance in complex coordination scenarios.
Abstract
Controlling the behavioral diversity is a pivotal challenge in multi-agent reinforcement learning (MARL), par- ticularly in complex collaborative scenarios. While existing methods attempt to regulate behavioral diversity by directly differentiating across all agents, they lack deep characteriza- tion and learning of multi-agent composition structures. This limitation leads to suboptimal performance or coordination failures when facing more complex or challenging tasks. To bridge this gap, we introduce Structured Diversity Control (SDC), a framework that redefines the system-wide diversity metric as a weighted combination of intra-group diversity, which is minimized for cohesion and inter-group diversity, which is maximized for specialization. The trade-off is governed by a pre-set Diversity Structure Factor (DSF), allowing for fine-grained, group-aware control over the collective strategy. Our method directly constrains the policy architecture without altering reward functions. This structural definition of diversity enables SDC to deliver substantial performance gains across various experiments, including increasing average rewards by up to 47.1% in multi-target pursuit and reducing episode lengths by 12.82% in complex neutralization scenarios. The proposed method offers a novel analytical perspective on the problem of cooperation in group-aware multi-agent systems.