Distributed FilterNet Reinforcement Learning for Achieving Output Consensus in Heterogeneous Multiplayer Multiagent Systems.

Abstract

We study the leader-follower consensus problem in multiagent systems with heterogeneous agent dynamics and multiple internal players per agent, each with distinct and interaffected objectives. Formulated as a multiplayer differential game per agent, the goal is to achieve output consensus among all agents while ensuring Nash equilibrium controls across each agent's internal players. To address this challenge, we introduce a distributed control framework that integrates both feedforward (regulator-based) and feedback (game-theoretic Riccati-based) components. We further design a FilterNet reinforcement learning (RL) architecture that solves the control solutions while eliminating the need for large-scale distributed data storage. Organized into four layers, FilterNet handles admissible policy identification, online initialization, asynchronous updates for Nash policy convergence, and real-time regulator solutions. This design reduces data requirements, ensures initial excitation, and accelerates convergence. Theoretical guarantees establish conditions for solvability and convergence. Numerical simulations and comparisons with existing methods confirm the effectiveness and superiority of the proposed approach.