Deterministic Convergence Analysis and Application of Elman Neural Network via Sparse Mechanism and Entropy Error Function.

Abstract

In this study, we employed the batch gradient method to investigate the monotonicity and convergence of the Elman neural network (ENN) based on the entropy error function (EEF) and regularization methods. This enhances network stability and sparsity while also boosting its ability to generalize. Traditional mean square error (mse) functions in complex networks often result in slower convergence during training, prone-to-local minima, and even incorrect saturation issues. To address this drawback, we propose a novel EEF for training ENN, effectively avoiding the problem of learning speed degradation. Furthermore, by leveraging smoothing group $L_{1/2}$ regularization $(\text {SGL}_{1/2})$ methods in studying ENN based on EEF, we effectively overcome the drawbacks of traditional group $L_{1/2}$ regularization $(\text {GL}_{1/2})$ leading to error function oscillations. In addition, we optimize the network architecture effectively in two key ways: reducing redundant nodes to near 0 and driving redundant weights toward 0 for remaining nodes, further boosting network sparsity. This article rigorously proves the monotonicity of the error function, alongside presenting strong and weak convergence outcomes for the novel method. The effectiveness and correctness of our approach are clearly illustrated through experimental results. The simulation results align with the theoretical findings.