Adaptively trigger memory network with temporal consistency for semi-supervised long video object segmentation.

Abstract

Semi-supervised video object segmentation is an extremely challenging task especially for long video sequence due to the difficulty in fully exploring the spatiotemporal information. Hereby, we propose a video object segmentation model based on adaptive memory bank and time consistency to build spatiotemporal relationship. First, we design an adaptive memory bank, in which the update trigger module can detect inter-frame differences and adaptively trigger memory bank updates, avoiding ignoring key frames and reducing redundant calculations on unrelated pixels. The feature compression and deletion mechanism in the memory bank prevents the unlimited expansion of the memory bank and reduces performance degradation. A temporal consistency module is then added to provide object location priors, complementing the lack of temporal locality. Extensive experiment demonstrates that our model is able to achieve accurate and stable segmentation for long videos.