Fractional-order gradient descent learning for Elman neural networks.

Abstract

To address the limitations of conventional integer-order gradient descent in training Elman neural networks, such as susceptibility to local minima and slow convergence-this paper proposes a fractional-order gradient descent learning algorithm for Elman networks based on the Grünwald-Letnikov definition (FO-Elman). First, the fractional-order gradient expressions for each layer of the Elman network are systematically derived, and a complete backpropagation framework is established; the convergence of the proposed algorithm is also proved theoretically. Then, by exploiting the memory property of fractional calculus, the proposed method incorporates a weighted aggregation of historical gradient information into the parameter update rule, thereby mitigating the shortcomings of standard optimization schemes. Finally, experimental results on system identification and time-series prediction tasks demonstrate that FO-Elman achieves improved optimization performance for Elman networks, providing a new theoretical and algorithmic tool for recurrent neural network training.