A Decomposition Principle for Complexity Reduction of Artificial Neural Networks.

Abstract

A decomposition principle is developed for systematic determination of the dimensionality and the connections of Hopfield-type associative memory networks. Given a set of high dimensional prototype vectors of given memory objects, we develop decomposition algorithms to extract a set of lower dimensional key features of the pattern vectors. Every key feature can be used to build an associative memory with the lowest complexity, and more than one key feature can be simultaneously used to build networks with higher recognition accuracy. In the latter case, we further propose a "decomposed neural network" based on a new encoding scheme to reduce the network complexity. In contrast to the original Hopfield network, the decomposed networks not only increase the network's storage capacity, but also reduce the network's connection complexity from quadratic to linear growth with the network dimension. Both theoretical analysis and simulation results demonstrate that the proposed principle is powerful. Copyright 1996 Elsevier Science Ltd