Discriminately Treating Motion Components Evolves Joint Depth and Ego-Motion Learning.

Abstract

Unsupervised learning of depth and ego-motion, two fundamental tasks in 3D perception, has made significant strides in recent years. Nevertheless, existing methods often treat ego-motion estimation as an auxiliary task, typically indiscriminately mixing all motion types when generating supervisory signals. Such designs hinder the incorporation of sufficiently strong geometric constraints into the joint learning framework, thereby limiting its robustness under diverse adverse conditions. This study introduces a discriminative treatment of motion components by leveraging the distinct geometric regularities inherent in their respective rigid flows, which simultaneously benefits both depth and ego-motion estimation. Given consecutive video frames, the network outputs are first employed to align the optical axes and imaging planes of the source and target cameras. Optical flows between the original frames are then transformed through these alignment processes, and their deviations are quantified to impose geometric constraints on each estimated ego-motion component. These alignments further reformulate joint learning into coaxial and coplanar forms, where depth and each translation component can be mutually derived through closed-form geometric relationships, thereby introducing constraints that improve the stability of depth learning. Extensive experiments on multiple public datasets and a newly collected, diverse real-world dataset demonstrate the superior performance of the proposed method, particularly under challenging conditions where existing approaches often fail. Our source code is publicly available at https://mias.group/DiMoDE.