Learn the interactions: Weakly supervised video anomaly detection with human-object interactions.

Abstract

Weakly supervised video anomaly detection remains a challenging problem, primarily due to the scarcity of abnormal training samples and the lack of diverse feature representations, which hamper the learning of discriminative models. To address these issues, we introduce a novel weakly supervised cross-domain framework that enriches feature representations by explicitly integrating semantic cues from Human-Object Interactions (HOI) and action dynamics. Specifically, we model domain-specific features using two independent encoders and fuse them via a bi-directional cross-attention mechanism to obtain a unified embedding. To further alleviate class imbalance and enhance feature discrimination, our network is jointly optimized with classification loss, a cross-domain contrastive loss, and a clip-level focal loss. The proposed method is evaluated with various visual backbones, demonstrating its adaptability and robustness. Extensive experiments show that our approach achieves competitive performance against state-of-the-art methods under weak supervision in terms of AUC. Specifically, our proposed method achieves 98.33% AUC and 87.96% AUC on two benchmark datasets, i.e., ShanghaiTech and UCF-Crime, respectively.