Blind signal processing by the adaptive activation function neurons.

Abstract

The aim of this paper is to study an Information Theory based learning theory for neural units endowed with adaptive activation functions. The learning theory has the target to force the neuron to approximate the input-output transference that makes it flat (uniform) the probability density function of its output or, equivalently, that maximizes the entropy of the neuron response. Then, a network of adaptive activation function neurons is studied, and the effectiveness of the new structure is tested on Independent Component Analysis (ICA) problems. The new ICA neural algorithm is compared with the closely related 'Mixture of Densities' (MOD) technique by Xu et al.. Both simulation results and structural comparison show the new method is effective and more efficient in computational complexity.