Online Value Iteration for Unknown Nonlinear Multiagent Systems: A Model-Decoupled Encoding-Decoding Mechanism.

Abstract

In practice, optimal consensus control for multiagent systems (MASs) is strictly constrained by limited communication bandwidth. Therefore, dynamic encoding-decoding mechanisms are designed to address this issue. However, unknown nonlinear dynamics inevitably generate uncertainties during the encoding-decoding process, which may lead to quantizer saturation and consequently affect consensus performance. To address this challenge, this article proposes an online reinforcement learning (RL) control strategy based on a novel model-decoupled dynamic encoding-decoding mechanism. Distinct from existing schemes relying on model parameters, the proposed mechanism features a model-decoupled architecture where the encoder and decoder structures are constructed without embedding explicit system dynamics. To handle the unknown system model, a novel online identifier is designed to actively compensate for dynamic uncertainties faced by the encoding-decoding mechanism, theoretically guaranteeing quantizer nonsaturation and ensuring data validity. Based on this reliable quantization communication, a distributed value iteration (VI) algorithm is developed to obtain optimal policies using solely decoded state information, eliminating the dependence on control policies of neighbors. Simulation studies on a heterogeneous UAV-UGV formation verify the robustness and effectiveness of the proposed method.