Temporal Mask-Embedding Learning and Query-Refined Head Network for Visual Tracking.

Abstract

Recent trackers aim to improve tracking performance by propagating temporal information across consecutive frames. However, they usually rely on a single compressed temporal token, which restricts information capacity and may lead to information loss. Moreover, they typically adopt a conventional tracking head network to directly regress fixed bounding box coordinates, neglecting both the ambiguity introduced by the temporal token during bounding box localization and the inherent uncertainty of the tracking task. To resolve those issues, we propose TMQRTrack, a novel visual tracking network, to improve tracking performance from two perspectives. In particular, we first propose an innovative temporal mask-embedding learning mechanism, which introduces a sufficient number of learnable temporal mask-embedding tokens to preserve rich environmental cues and comprehensively perceive the target object. Particularly, these mask-embedding tokens perform mask-embedding learning via the mask-guided attention and are then efficiently propagated across frames by our temporal propagation module (TPM). Then we propose a novel query-refined head network to alleviate the localization ambiguity potentially induced by the temporal tokens and explicitly model the uncertainty of the tracking task. Our head network reformulates the bounding box regression task into a probabilistic learning problem, predicting bounding box offsets based on learned distributions and iteratively refining each side of the box in successive iterations. Extensive experiments on short- and long-term tracking benchmarks, including GOT-10k, LaSOT, $\text{LaSOT}_{ext}$ , and TrackingNet, demonstrate that our TMQRTrack can achieve state-of-the-art performance.