Alleviating noise memorization for adversarially robust few-shot learning.

Abstract

Few-Shot Learning (FSL) enables models to learn from just a few examples of new classes by leveraging knowledge from base classes. While FSL has made significant strides, its vulnerability to adversarial attacks-especially with limited data-has been overlooked. To address this, adversarial training is often used to build more robust models. However, we found that this approach can lead the model to memorize adversarial noise, which harms its ability to generalize. Hard labels exacerbate this issue by pushing the model toward perfect accuracy on adversarial examples, while also making it less robust to small changes in weights. To solve these problems, we propose Alleviation of Noise Memorization (ANM), a method that includes Adaptive Label Smoothing for more flexible supervision and Robust Weight Learning to enhance model stability. Our extensive experiments show that ANM effectively reduces noise memorization and improves generalization, outperforming current benchmarks.