Boosting Black-Box Unsupervised Domain Adaptation With Cross-Model Contrastive and Consistency Learning.

Abstract

Black-box unsupervised domain adaptation (black-box UDA) aims to adapt a target model without accessing source data or source-model parameters. Compared with conventional unsupervised domain adaptation (UDA) and source-free UDA (SFUDA), this setting better protects data privacy and model security, but poses greater challenges for knowledge transfer, especially for lightweight target networks with limited learning capacity. To address this problem, we propose a novel framework called cross-model contrastive and consistency learning (CMCCL) for black-box UDA tasks. CMCCL first introduces an asymmetric dual-network collaboration mechanism, where a lightweight primary model and an auxiliary model provide complementary supervision. It then incorporates mixed and masked images into cross-model consistency learning (CMCL) to exploit both pairwise structure present in target data and spatial relationships of individual objects within images. Furthermore, CMCCL develops cross-model contrastive learning with interpolation-based and global-local self-supervised objectives to improve representation discriminability and robustness. Finally, we extend CMCCL to CMCCL++ by distilling knowledge from off-the-shelf vision-language (ViL) models, leveraging their rich semantic information to guide the adaptation process. Extensive experiments on four challenging cross-domain benchmarks demonstrate that our method achieves state-of-the-art performance with lightweight target models. Notably, it even surpasses numerous white-box SFUDA methods employing cumbersome backbones.