Self-gating stochastic-resonance-based autoencoder for unsupervised learning.

Abstract

Incorporating additive noise components to an ensemble of McCulloch-Pitts neurons can enhance the information representation of the input, asymptotically approaching the average firing probability for large enough ensembles. We further multiply the input by the average firing probability to control the higher probability of self-gating, thereby forming a unified noise-boosted activation model with learnable noise-related hyperparameters. This gating strategy plays a crucial role in improving the performance of neural networks, as evidenced by the optimization of the autoencoder loss at nonzero optimal-noise-scaling hyperparameters, a phenomenon termed self-gating stochastic resonance. Experiments with designed autoencoders using noise-boosted activation functions demonstrate the potential applications of the self-gating stochastic resonance effect in the field of unsupervised learning.