Batch self-organizing memory neural network for continual supervised learning.

Abstract

Continual learning enables artificial neural networks to learn new tasks without forgetting previously learned tasks, which is a key challenge in mimicking human intelligence. Dynamic architecture strategies are therefore employed to expand the capacity of deep networks, enabling them to incorporate new tasks while maintaining performance on previously learned ones. However, deep networks frequently struggle to select an appropriate network expansion strategy. Task identifiers, manually assigned to distinguish tasks, are often required to guide parameter or component selection for each task. To address these challenges, this paper proposes a Batch Self-organizing Memory Neural Network (Batch SOMNN) for continual supervised learning. First, a Batch Supervised Competitive Learning (BSCL) algorithm is proposed for competitive learning of new tasks without task identifiers. By identifying distributional shifts between new and old tasks, the algorithm dynamically generates new regions to accommodate new data while preserving existing memory. Second, a Memory Correction (MC) module is employed to selectively retain valuable information by utilizing the forgetting curve and an adaptive threshold matrix, which enhances the efficiency and cost-effectiveness of network expansion. Extensive experiments show that Batch SOMNN outperforms strong baselines in continual learning scenarios. To further enhance its scalability and performance on complex datasets, we extend Batch SOMNN with a deep feature extraction backbone and prototype-based classification, forming Deep Batch SOMNN (DB-SOMNN), which achieves state-of-the-art results on standard benchmarks.