Semantic-guided compositional scene representation framework.

Abstract

Neural scene representation methods play a key role in computer vision and graphics, but it is difficult for them to generalize to new invisible scenes. Addressing this problem, we propose a semantic-guided compositional scene representation framework in this paper, consisting of a baseline module for scene representation, a semantic mapping module, and a compositional representation strategy. In the proposed framework, the semantic mapping module learns an embedding correlation by training on visible scenes, where the embedding correlation maps explicit semantic attributes to implicit scene representations. By utilizing the embedding correlation, the framework can represent invisible scenes using only semantic attributes. Besides, the compositional representation strategy is designed to fuse the decomposed 1-object scene representations into multi-object scene representation, yielding a higher training efficiency for multi-object scenes. Extensive experimental results on three datasets demonstrate that the proposed framework can achieve high-accuracy representations for visible and invisible multi-object scenes.