Decoupling geometry and appearance in Gaussian splatting for reflective surface reconstruction: A glossy image prior-guided approach.

Abstract

Accurate 3D reconstruction of real-world scenes is fundamental for applications in robotics and virtual reality. However, the view-dependent highlights of the ubiquitous presence of reflective and glossy surfaces violate the photometric consistency assumption central to most reconstruction algorithms, leading to severe geometric and appearance artifacts. Therefore, we introduce GIP-GS, a novel approach that achieves high-fidelity reconstruction of objects with significant specular reflections through a principled, two-stage strategy that decouples geometry optimization from appearance refinement. First, we introduce a Color-space Analysis based Reflective Region Detection (CA-RRD) module to generate a continuous Glossy Image Prior (GIP), which flags photometrically inconsistent (specular) areas versus consistent (diffuse) ones across views. With this prior, our region-differentiated geometric optimization applies tailored supervision: a novel connected region normal consistency loss mitigates artifacts in specular areas, while priors from pre-trained networks enforce fidelity in diffuse regions, establishing a robust geometric scaffold. Second, to surmount the expressive limitations of Spherical Harmonics, we propose a dedicated appearance refinement stage. This stage introduces a novel gated appearance fusion network(GAFN) guided by the GIP, which explicitly disentangles base color and specular effects. To further enhance fidelity, this network is conditioned on a Wavelet-based Multi-scale Feature Extraction, enabling the accurate rendering of complex, high-frequency specular highlights. We evaluate GIP-GS on challenging synthetic (e.g., NeRF Synthetic, Shiny Blender) and real-world (e.g., DTU) datasets. Our method achieves competitive performance.