Multilayer neural networks and Bayes decision theory.

Abstract

There are many applications of multilayer neural networks to pattern classification problems in the engineering field. Recently, it has been shown that Bayes a posteriori probability can be estimated by feedforward neural networks through computer simulation. In this paper, Bayes decision theory is combined with the approximation theory on three-layer neural networks, and the two-category n-dimensional Gaussian classification problem is studied. First, we prove theoretically that three-layer neural networks with at least 2n hidden units have the capability of approximating the a posteriori probability in the two-category classification problem with arbitrary accuracy. Second, we prove that the input-output function of neural networks with at least 2n hidden units tends to the a posteriori probability as Back-Propagation learning proceeds ideally. These results provide a theoretical basis for the study of pattern classification by computer simulation.