High Frame Rate iToF-Flow-Based Depth Imaging Enabled by Local Linear Transfer.

Abstract

iToF is a prevalent, cost-effective technology for 3D perception. However, its reliance on multiple measurements commonly leads to reduced performance in dynamic scenes. Building on our observation of local linear transfer (LLT) characteristics and the analysis of the physical iToF imaging process, we propose the generalized iToF-flow (GiF) model, consisting of cross-mode and uni-mode flow, to account for variations caused by different measurement modes and 3D motion. We design a generalized iToF-flow-based depth extraction network (GiFDEN), featuring an integrated LLT-based cross-mode transfer module (ILCTM) for mode-varying and pixel shift compensation of cross-mode flow and a uni-mode photometric compensation module (UPCM) for photometric residual caused by depth-wise motion in uni-mode flow. The proposed network enables accurate estimation of the four-phase measurements at each time step, thereby achieving high-frame-rate, accurate depth retrieval. Extensive experiments on simulated and real-world data demonstrate the effectiveness of the proposed method. Compared with the SOTA method, our approach reduces the computation time by 87$\%$ and depth reconstruction error by 55$\%$.