Self-action distribution system (SADS): From neural network stability analysis to memory networks with low storage space cost.

Abstract

Currently, neural network code editing is employed directly, without any theoretical analysis of its stability. This practice can lead to network instability and unnecessary resource consumption. To address these issues, this paper makes the following innovative contributions. First, we introduce a method for evaluating the stability of neural networks. Since this evaluation does not require gradient computation, it is applicable to a wide range of complex architectures. By applying this method, algorithm optimization can be performed prior to training, which helps identify potential performance bottlenecks at an early stage. Second, building on the proposed evaluation method, we present an implementation of a memory network with favorable space complexity. Both theoretical analysis and experimental results confirm that this memory network can effectively represent symbols. Interestingly, the proposed memory mechanism aligns with certain functions of the human brain's temporal lobe, offering valuable insights for research areas such as brain-inspired computing and the development of in-memory computing chips. Finally, based on this memory network, we propose a novel neural network algorithm called the Self-Action Distribution System (SADS). Experimental validation demonstrates that SADS outperforms existing algorithms, confirming its effectiveness.