Robust Reconstructed Neural Network Based on Spectral Elastic Activation.

Abstract

The presence of sparse samples poses a formidable challenge for neural networks (NNs) in shaping representative patterns due to the limited coverage of concentrative activation range in NNs. To address this issue, a robust reconstructed NN, based on spectral elastic activation (SEA-RRNN), is developed in this article. Primarily, a spectral elastic activation (SEA) is designed to broaden the original activation of NNs, which embeds a spectral increment scaled by the estimated outlier degree on the activation boundary. It enables SEA-RRNN to stretch the boundary of SEA to cover the features of sparse samples. Then, an adaptive robust gradient descent (ARGD) algorithm is introduced to update the parameters of SEA. By tuning the loss between error and correntropy with the estimated outlier degree, the ARGD algorithm establishes two loss functions with complementary distance sensitivity for different parameters of SEA, which alleviates the construction conflict of robust center and precise boundary in SEA-RRNN. Furthermore, the theoretical analysis of SEA-RRNN is provided to validate its convergence and robustness. Finally, the experimental results demonstrate that SEA-RRNN exhibits superior robustness compared to other NN models.