Knowledge Diffusion-Based Adaptive Alignment with Hierarchical Context for Video Temporal Grounding.

Abstract

Video Temporal Grounding (VTG) aims to localize the temporal moment that corresponds to a natural-language query in an untrimmed video. However, VTG faces two core challenges: narrowing the semantic gap between concise text queries and complex visual content, and extracting discriminative moment cues from long background-dominant sequences. To address these challenges, we propose Knowledge Diffusion Alignment (KDA), a diffusion-based framework that operates in the semantic feature domain. At its core, a Background Residual Diffusion Model (BRDM) starts from adaptive global video knowledge and, under query conditioning, progressively removes query-irrelevant background residuals to synthesize moment knowledge. In addition, to provide a strong conditional prior and multi-scale supervision, Hierarchical Video Knowledge Extraction (HVKE) aggregates context at multiple temporal granularities, producing both video and moment knowledge. KDA is trained in three stages: first learning target-aware moment knowledge, then aligning the diffusion process to that target, and finally performing end-to-end inference using BRDM-generated knowledge. The synthesized moment knowledge is injected into knowledge prompt reasoning, where a Dynamic Moment-aware Encoder (DME) adaptively generates dynamic temporal convolution kernels conditioned on the moment knowledge, coupling efficient local modeling with global self-attention. For prediction, a Spans-Enhanced Hierarchical Decoder (SEHD) constructs a temporal pyramid and adaptively selects the most relevant initial spans to exploit intrinsic alignment cues. Experiments across public benchmarks demonstrate that the proposed KDA outperforms the state-of-the-art methods, and extensive ablations further verify the effectiveness of the module design. The code is available at https://github.com/Evangelion09/HKDA.