Multi-agent self-attention reinforcement learning for multi-USV hunting target.

Abstract

A reinforcement learning (RL) method based on the multi-head self-attention (MSA) mechanism is proposed to solve the challenge of multiple unmanned surface vehicles (multi-USV) hunting target at the surface. The kinematic, dynamic, and environmental models of the USV are established. The rules and constraints for successful hunting are defined. A differential game model is developed for multi-USV hunting target, detailing the cooperative dynamics among the multi-USV and their competitive interactions with the target. The action space, state space, and reward function are also designed. A multi-agent reinforcement learning (MARL) algorithm incorporating MSA is introduced. The method pays more attention to the key information and extremely improves the convergence speed of the algorithm. According to the experimental results, the convergence speed and hunting success rate of the proposed method are improved by 17% and 8% compared with the baseline-based MARL algorithm, respectively.