Online lifelong optimal tracking control of uncertain nonlinear continuous-time strict-feedback systems using deep neural networks.

Abstract

A novel integral reinforcement learning (IRL)-based optimal trajectory tracking scheme for nonlinear continuous-time systems in strict feedback form is introduced by using backstepping and multilayer or deep neural networks (DNNs). The proposed method employs a dynamic surface control-based technique in an optimal framework to relax the need for repeatedly computing the derivatives of virtual controllers at each step of the backstepping process. An online singular value decomposition (SVD)-of the activation function gradient-based actor-critic DNN at each step of the backstepping process is employed to minimize a discounted value function. Novel online SVD-based weight update laws, which are shown to mitigate vanishing gradient, for the actor and critic DNNs are derived by using control input error and Bellman error respectively. A new online lifelong learning (LL) technique using Bellman residual and control input errors to overcome the issue of catastrophic forgetting in both critic and actor DNNs is also attempted, and closed-loop stability is analyzed and demonstrated. The effectiveness of the proposed method is shown in simulation on mobile robot tracking and ship autopilot, which demonstrates a 76% total cost reduction when compared to the literature.