Remote sensing object detection through hierarchical feature mining and multivariate head collaboration with knowledge distillation.

Abstract

Knowledge distillation (KD) is a proven technique for enhancing the performance of lightweight models in intelligent edge applications for remote sensing. However, existing KD approaches often fall short in fully leveraging the statistical information embedded within feature maps and tend to neglect the potential benefits of coupling multiple teacher-student detection heads. To overcome these limitations, this paper introduces a novel KD framework-Hierarchical Feature Mining and Multivariate Head Collaboration (HMKD)-designed to enhance lightweight model performance through effective information extraction and structural collaboration. The proposed method includes two key modules: Low-Level Feature Distillation for Distributed Information Mining (LFDIM) and High-Level Feature Distillation for Extraction of Channel Semantic Knowledge (HFECS). These modules target distinct feature layers to extract meaningful statistical information, effectively narrowing the information transfer gap between teacher and student models. Additionally, the Collaboration Distillation of Multivariate Head (CDMH) module is introduced to facilitate comprehensive interaction among multiple teacher-student detection heads. This module enables the concurrent transfer of both classification and regression knowledge, thereby addressing target conflicts and capturing latent relationships within region-based features. Extensive experiments on two publicly available remote sensing datasets, DOTA and DIOR, demonstrate that HMKD significantly improves detection performance across both single-stage and two-stage lightweight models. These results validate the method's effectiveness and adaptability across diverse remote sensing scenarios.