Towards understanding memory buffer based continual learning.

Abstract

Continual learning (CL) enables models to adapt to and retain knowledge across a sequence of tasks. While recent theoretical studies primarily focus on CL without memory buffers, the theoretical analysis of memory buffer-based continual learning (MCL), a widely adopted approach in CL, remains underexplored. To bridge this gap, we propose a theoretical framework to characterize the impact of memory buffers in CL. Notably, our findings reveal that incorporating a memory buffer does not inherently guarantee reduced forgetting or improved generalization compared to training without one (NCL), potentially resulting in inefficient memory usage and unnecessary computation. Specifically, we derive expressions for expected forgetting and generalization errors in overparameterized linear models across an arbitrary number of tasks. Furthermore, we provide a coefficient by coefficient comparison that clarifies when and why MCL helps or hurts relative to NCL. In particular, when tasks are similar, enlarging the memory buffer should be accompanied by increasing model capacity, which yields lower forgetting. By contrast, when tasks are highly dissimilar, MCL may forget more and generalize worse than NCL. Finally, we validate our theoretical insights through experiments with deep neural networks, providing practical guidance for algorithm design in MCL.