Depth map super resolution grounded in the multi-dimensional collaborative guidance.

Abstract

The depth map super-resolution guided by RGB images aims to reconstruct a high-resolution (HR) depth from a low-resolution (LR) depth map input with the assistance of a corresponding HR RGB image. Convolutional methods often struggle to maintain global consistency due to limited receptive fields, while Transformer-based models incur high computational costs on HR data. Furthermore, naive cross-modal fusion strategies frequently introduce texture copying artifacts or edge blurring when the RGB and depth modalities are misaligned. To tackle these challenges, we propose a multi-dimensional collaborative guidance network (MCGNet), a novel framework that combines the efficiency of State Space Models (SSM) with multi-dimensional guidance. First, we construct a spatial reconstruction module adopting the SSM as the reconstruction backbone. This module exploits the linear complexity of SSMs to model long-range dependencies, ensuring structurally consistent depth recovery across large smooth regions without the heavy burden of self-attention. Second, we introduce a spatial-frequency cooperative guidance module that leverages the Fourier transform to fuse RGB and depth features in the frequency domain, enabling explicit utilization of high-frequency RGB details to refine depth edges while preserving low-frequency depth structures. Third, an LR-HR mapping feature guidance module is developed to bridge the resolution gap, injecting scale-aware RGB priors into the depth decoding stream to facilitate precise LR-HR mapping learning. Extensive experiments on benchmark datasets demonstrate that our approach achieves state-of-the-art performance quantitatively and qualitatively.