DeepART: Deep gradient-free local learning with adaptive resonance.

Abstract

This article presents DeepART, a gradient-free and one-shot incremental learning technique for the training of deep Hebbian neural networks using the dynamics of Adaptive Resonance Theory (ART) algorithms. In the presented model, layers of a deep neural network are interpreted as modified FuzzyART modules with complement coded inputs and winner-take-all local weight updates with a FuzzyARTMAP head module providing feature-category-label mapping to enable supervised learning. These local weight update rules are derived for fully-connected and convolutional layers. DeepART provides a performance boost, reduction of category proliferation, and subsequently improved scalability to comparable ART-based methods for processing high-dimensional datasets in lifelong learning scenarios, combining the nonlinear feature representation learning of deep neural networks with the one-shot and lifelong learning properties of ART algorithms.