GCM: Interpretable Multiagent Reinforcement Learning via Graph Cooperation Modeling.

Abstract

Multiagent reinforcement learning (MARL) has been widely investigated, ranging from theoretical analysis to real-life applications. However, the utilization of existing non-transparent neural network architectures has resulted in opaque decision-making processes, making it difficult for humans to understand and trust the models being used. Fundamentally, all data is a topological structure, which provides reliable transparency for MARL tasks due to its powerful relational expression capability, scalability, and explicit structural relationships. In this article, we propose a novel approach of graph cooperation modeling (GCM), explicitly capturing and comprehending the complex dynamics of collaborative relationships among agents with the graph structure. GCM learns a metric function to discern beneficial interactions among agents, integrating it into the agent aggregation strategy of a graph neural network (GNN) capable of modeling arbitrary-order interactions. Furthermore, GCM utilizes identity semantics together with global state and individual value functions to estimate the credit of each agent, enhancing each agent's distinct focus on task-related regions. Extensive experiments on a range of challenging MARL benchmarks demonstrate that GCM not only delivers up to 28.75% relative performance gains on super-hard maps but also offers clear interpretability that provides insights into the underlying cooperative patterns.