Semantic segmentation in adverse scenes with fewer labeled images.

Abstract

Images captured in adverse scenes suffer from degradation, blurring, and other issues, which are challenging to high-level vision tasks such as semantic segmentation. Labeling plenty of images in adverse scenes for the semantic segmentation task is time-consuming and user-unfriendly. In this study, we explore degraded image semantic segmentation by the semi-supervised paradigm to alleviate the demand for labels. However, traditional semi-supervised methods typically require a significant proportion of labeled images (12.5%, 25%, or 50% of total training images), which are still difficult to acquire in domains characterized by degradation. Our approach, in contrast, operates with significantly fewer labeled images (as little as 0.5%-1% of total training images). To achieve that, we propose a novel two-step training pipeline that separately handles the training of labeled and unlabeled images to prevent overfitting. In the first step, focused on labeled images, we introduce a re-parameterization domain adapter (RPDA) that facilitates efficient domain adaptation. In the second step, which targets unlabeled images, we employ a teacher network that distills knowledge acquired in the first step. Additionally, we develop a class adaptive threshold with label perception (CATLP) to generate accurate pseudo-labels. Extensive experiments on public datasets, including various adverse scene images, demonstrate the superiority of our method over the state-of-the-art methods. The code will be made publicly available.