Prototype-Based Meta-Prompt Tuning: Toward Rehearsal-Free Few-Shot Class-Incremental Learning for Multimodal Remote Sensing Image.

Abstract

Recent research on the joint classification of multimodal remote sensing data has achieved outstanding performance in tasks within predefined label spaces. However, surface conditions are dynamic and change over time, resulting in variations in land cover classes collected from the same region at different time points. As a result, when new classes are discovered, the previous works must use a combination of old and new class data to retrain the model, which incurs high computational costs and raises concerns about data privacy. In this work, we propose the prototype-based meta-prompt tuning (PMPT) framework, which fine-tunes only a few session-relevant visual prompts to adapt to incremental classes, while simultaneously learning prototype embeddings for each class to preserve historical knowledge. Specifically, the PMPT consists of a meta-learning-based feature representation backbone and an incrementally updated nearest-class-mean (NCM) classifier. The backbone is trained on base class data to learn shared and stable global knowledge, then frozen, with only the prompts fine-tuned to extract sessions-specific local knowledge from incremental sessions. The NCM classifier is a globally shared classifier that measures the similarity between test samples and prototypes, effectively alleviating the issues of knowledge forgetting and overfitting. Additionally, we propose an incremental prototype contrastive loss to reduce semantic drift and prototype overlap in the embedding space. During the testing phase, the PMPT reproduces the complete embedding function by matching samples, class prototypes, and visual prompts, thereby enabling accurate classification of unknown samples. The method has been tested on widely used multimodal remote sensing datasets, demonstrating the effectiveness of the proposed PMPT in addressing the dilemma of stability-plasticity with limited incremental samples. The code is available at https://github.com/Jiahuiqu/PMPT.