Learning Shape Anchors for Holistic Indoor Scene Understanding.

Abstract

This paper presents a shape anchor guided learning strategy (AncLearn) for robust holistic indoor scene understanding. We observe that the search space constructed by current methods for proposal feature grouping and instance point sampling often introduces massive noise to instance parsing and object reconstruction. Accordingly, we develop AncLearn to learn the shape anchors of 3D objects, to provide shape constraints in a top-down manner. The learned anchors fit to instance surfaces and thus provide fine localization clues for (i) separating noise and object-related features to offer reliable instance proposals, (ii) reducing outliers in object point sampling for providing well-structured geometry priors for object reconstruction and (iii) integrating RGB information at the instance level to incorporate additional texture clues for better semantic perception. We embed AncLearn strategy into a reconstruction-from detection learning system (AncRec++) to generate high-quality semantic scene models in a purely instance-oriented manner. Within AncLearn, the network learns to abstract the shape priors of each instance and thus improves its robustness against the sparsity and incompleteness of point clouds. Experiments conducted on the challenging ScanNetv2 dataset demonstrate that our method consistently achieves state-of-the-art performance in terms of 3D object detection, layout estimation, and shape reconstruction. Code is available at https://github.com/Geo-Tell/AncRec.