CMFF: Cross-modal feature fusion network for robust point cloud completion.

Abstract

In the field of 3D vision, point cloud data often suffers from partial missing regions due to occlusion, reflection, or viewpoint limitations, which severely affects downstream tasks. Existing cross modal point cloud completion methods directly employ cross attention mechanisms for feature fusion, without considering the feature distributions and noise between different modalities, leading to suboptimal completion results. In this paper, we propose a novel cross modal point cloud completion framework, CMFF, which takes partial point clouds and single view images as inputs. Specifically, firstly, it uses a point cloud encoder and an image encoder to extract features from the input. In the point cloud encoder, we propose a differential point transformer module for extracting local geometric details and global structural features of the point cloud, which enhances the representation and robustness of complex geometries. Second, we propose a differential cross transformer module for feature fusion. The redundant and conflicting cross modal features are filtered by differential operations to enhance the correlation of cross modal features and improve the completion accuracy. Third, coarse point cloud is generated using a point cloud patch generator. Finally, we propose the fine point cloud module, which optimizes multi modal features using simple attention mechanism and generates an offset vector to optimize the point cloud. Extensive experiments on the view-guided point cloud completion benchmark ShapeNet-ViPC and the Terracotta Warriors dataset show that CMFF outperforms 15 current methods to state-of-the-art on several point cloud completion metrics, exhibiting excellent performance and generalization ability.