HSFF-Net: Hierarchical spectral-feature fusion network for deepfake detection and localization.

Abstract

The rapid development of deepfake techniques poses a serious threat to multimedia authenticity, driving increased attention to deepfake detection. However, most existing methods focus solely on classification while overlooking forgery localization, which is essential for understanding manipulation intent. To address this issue, we propose a novel Hierarchical Spectral-Feature Fusion Network (HSFF-Net) for deepfake detection and localization from spatial- and frequency-domain views. Specifically, the Spectral Detail Amplification (SDA) module enhances tampering cues around facial features in the frequency domain. The Dynamic Collaborative Fusion (DCF) unit integrates complementary dual-stream features across multiple hierarchical levels to highlight valuable information. The Adaptive Feature Elevation (AFE) module bridges coarse semantic and fine-grained details in a top-down manner. Furthermore, the Global Guidance Exposure (GGE) module injects localization cues across feature levels to improve forgery localization accuracy. Additionally, we design the contrastive clustering loss for the detection task, which guides features to cluster around their corresponding class centers while simultaneously pushing them away from other class centers, thereby promoting intra-class compactness and inter-class separability. Abundant experiments demonstrate that HSFF-Net achieves superior performance on both detection and localization tasks, with good generalization across diverse datasets and robustness against various perturbations.