MtvTrack: Robust Visual Tracking via Modeling Time-Variant State of the Target.

Abstract

Current single-object tracking algorithms depend on the information supplied by the template to identify and locate the object within the search area. However, environmental complexities and unknown factors can alter the object's state, causing mismatches in template information. The existing works using the template update mechanism (TUM) and multiple template feature fusion have the following problems: 1) TUM is affected by input superposition, making it hard to eliminate noise; 2) they suffer a temporal lag in their responsiveness to changes that occur in the object during the tracking process; 3) it is insufficient to rely solely on visual features within the search area of the current frame to improve the template; and 4) the prior knowledge regarding the input is not fully leveraged to learn the time-variant state of the object. We observe that in complex tracking scenarios, humans subconsciously analyze the evolutionary patterns of the object and its surroundings and integrate this information with the object's initial impression, thereby maintaining an awareness of the object's temporal state. Motivated by this, we propose a novel solution to the above problem, named MtvTrack, which can model the time-variant state of the object through the dynamic evolution pattern and static initial impression. Simultaneously, we propose a method for predicting the evolution pattern of scenes by utilizing past, present, and future (PPF) states. This approach effectively eliminates the information redundancy between consecutive frames and addresses the issue of delayed predictions of the target state in relation to changes within the search area. We establish a joint probability generative model and fully utilize prior knowledge to learn the time-variant state of the object. In addition, we develop a vector quantized-PPF (VQPPF) module for predicting the object's time-variant state. Experimental results on public benchmarks confirm the superior performance of our method. Source code is available at: https://github.com/long-wa/MtvTrack-main.