Q-Learning-Based Robust Control for Nonlinear Systems With Mismatched Perturbations.

Abstract

This brief presents a novel optimal control (OC) approach based on $\mathcal {Q}$ -learning to address robust control challenges for uncertain nonlinear systems subject to mismatched perturbations. Unlike conventional methodologies that solve the robust control problem directly, our approach reformulates the problem by minimizing a value function that integrates perturbation information. The $\mathcal {Q}$ -function is subsequently constructed by coupling the optimal value function with the Hamiltonian function. To estimate the parameters of the $\mathcal {Q}$ -function, an integral reinforcement learning (IRL) technique is employed to develop a critic neural network (NN). Leveraging this parameterized $\mathcal {Q}$ -function, we derive a model-free OC solution that generalizes the model-based formulation. Furthermore, using Lyapunov's direct method, the resulting closed-loop system is guaranteed to have uniform ultimate bounded stability. A case study is presented to showcase the effectiveness and applicability of the proposed approach.