Reinforcement Learning in Pursuit-Evasion Differential Game: Safety, Stability, and Robustness.

Abstract

Safety and stability are two critical concerns in pursuit-evasion (PE) problems in an obstacle-rich environment. Most existing works combine control barrier functions (CBFs) and reinforcement learning (RL) to provide an efficient and safe solution. However, they do not consider the presence of disturbances, such as wind gust and actuator fault, which exist in many practical applications. This article integrates CBFs and a sliding mode control (SMC) term into RL to simultaneously address safety, stability, and robustness. However, this integration is significantly challenging due to the strong coupling between the CBF and SMC terms. Inspired by the Stackelberg game, we handle the coupling issue by proposing a hierarchical design scheme where SMC and safe control terms interact with each other in a leader-follower manner. Specifically, the CBF controller, acting as the leader, enforces safety independently of the SMC design; while the SMC term, as the follower, is designed based on the CBF controller. We then formulate the PE problem as a zero-sum game and propose a safe, robust RL framework to learn the min-max strategy online. A sufficient condition is provided under which the proposed algorithm remains effective even when constraints are conflicting. Simulation results demonstrate the effectiveness of the proposed safe, robust RL framework.