Stochastic style perturbation modelling for visible-Infrared person re-Identification with severely modality imbalance.

Abstract

In this paper, we address the challenge of data imbalance in visible-infrared person re-identification (VI-ReID). Previous studies often presuppose a uniform distribution of training data across various modalities, however, due to constraints such as device limitations, privacy concerns, and operational conditions, gathering nightime infrared modality data can be prohibitively expensive or even impossible. Consequently, the limited infrared (IR) modality data tend to be overshadowed by the more plentiful visible (RGB) modality data during the training, particularly in scenarios marked by significant imbalance. To counter this issue, we introduce the Co-Modality Balance Learning (CMBL) framework, designed to recalibrate the balance in cross-modality learning and enhance the extraction of discriminative features. Initially, we design a Stochastic Style Perturbation (SSP) module that dynamically generates IR modality samples within the deep feature space to emulate the characteristics of a balanced dataset. Subsequently, we develop a cross-distribution alignment loss, which enables a refined optimization of sparse modality features to improve their accuracy and robustness. Additionally, we propose the novel Class-Aware Contrast Similarity Learning (CACS) strategy, which capitalizes on latent feature consistency to boost intra-class compactness and inter-class separation. Our extensive empirical evaluations and ablation studies on two publicly available cross-modality datasets under imbalanced conditions underscore the efficacy of our approach, showcasing its ability to adeptly navigate the complexities of data imbalance in VI-ReID.