UTN: Unsupervised optical flow estimation network based on transformer.

Abstract

With the aim of enabling unsupervised optical flow estimation, we propose a scalable framework based on a transformer and a feature pyramid network (FPN). Central to our approach is the incorporation of a transformer-CNN based structure within the encoder, designed to capture global and local dependency features from input image pairs-a crucial element for precise pixel-wise flow estimation. Subsequently, we integrate a normalized cross-correlation module (NCCM) and an attention-based intermediate flow estimation (AIFE) module into the FPN-based decoder. The NCCM enhances the decoder's focus on the saliency of shared foreground objects through correlation operations, while the AIFE refines flow estimation using an auxiliary positional mask and intermediate flow matrix. Furthermore, we propose a static optical flow loss, providing a distinct training clue that effectively boosts flow accuracy. Comprehensive experiments, including comparisons with state-of-the-art methods and ablation studies, were conducted across benchmark datasets such as FlyingChairs, MPI-Sintel, KITTI-2012, and KITTI-2015. Notably, our method achieved substantial performance gains. For instance, on the MPI-Sintel dataset, we observed a reduction in End-Point-Error (EPE) of 24.27 % on the clean dataset and 28.01 % on the final dataset compared to ARFlow. Ablation studies corroborated the efficacy of the NCCM, AIFE, and static optical flow loss in enhancing estimation accuracy.