A graph-based safe reinforcement learning method for multi-agent cooperation.

Abstract

Safety and Restricted Communication are two critical challenges faced by practical Multi-Agent Systems (MAS). However, most Multi-Agent Reinforcement Learning (MARL) algorithms that rely solely on reward shaping are ineffective in ensuring safety, and their applicability is rather limited due to the fully connected communication. To address these issues, we propose a novel framework, Graph-based Safe MARL (GS-MARL), to enhance the safety and scalability of MARL methods. Leveraging the inherent graph structure of MAS, we design a Graph Neural Network (GNN) based on message passing to aggregate local observations and communications of varying sizes. Furthermore, we develop a constrained joint policy optimization method in the setting of local observation to improve safety. Simulation experiments demonstrate that GS-MARL achieves a better trade-off between optimality and safety compared to other methods, and in large-scale communication-limited scenarios GS-MARL achieves a success rate at least 10% higher than the leading baselines. The feasibility of our method is also verified by hardware implementation with Mecanum-wheeled vehicles. Codes and demos are available at https://github.com/finleygou/GS-MARL.