Dual-domain self-supervised feature alignment via spectral-spatial representation learning for deepfake anomaly detection.

Abstract

Deepfake anomaly detection has become increasingly critical in visual security applications. However, many existing methods rely heavily on large-scale supervised forgery annotations and often struggle to capture subtle manipulation traces distributed across both spatial and frequency domains. To address these limitations, we propose a Self-Supervised Dual-Domain Alignment (SDDA) framework that jointly learns spatial and spectral representations without requiring labeled forged samples. Specifically, SDDA employs two parameter-sharing encoders to extract complementary features from RGB images and their frequency-domain counterparts, while a cross-domain alignment module enforces consistency between spatial textures and spectral signatures. To provide explicit and reproducible self-supervised supervision, four complementary descriptors are further derived from FAN feature maps: Local Structural Deviation (LSD) for local gradient inconsistency, Global Pattern Discrepancy (GPD) for holistic channel-correlation differences, Local Residual Difference (LRD) for residual-level inconsistency, and Total Consistency Deviation (TCD) for multi-layer feature deviation. These descriptors are predicted from the fused spatial-frequency embedding, encouraging the model to learn manipulation-sensitive representations in the absence of fake samples. In addition, a dual-domain contrastive objective enhances the discrimination of subtle forgery-related anomalies while improving robustness against real-world degradations such as compression and illumination variations. Experimental results demonstrate that SDDA consistently outperforms state-of-the-art baselines, particularly under challenging unseen-manipulation scenarios. Overall, the proposed framework provides an interpretable and label-efficient solution for dual-domain deepfake anomaly detection.