RGS: Reflection-Aware Gaussian Splatting Via Learning Geometry Continuity for Reflective Objects
Xiaobiao Du, Yida Wang, Cheng Bi, Kun Zhan, Yu Xin
AI summary
Problem
Existing 3D Gaussian Splatting methods suffer from geometric collapse and inaccurate specular rendering on reflective surfaces due to view-dependent reflection ambiguity and the inability of traditional depth/normal estimators to handle high-gloss regions.
Approach
The authors introduce a physically-based deferred rendering framework that leverages a 3D foundation model for cross-view geometry regularization and a reflection-guided densification strategy to precisely capture and render specular variations.
Key results
- Physically-based deferred rendering with learnable HDR and BRDF modeling
- Cross-view shape consistency regularization using VGGT to eliminate geometric hollows
- Reflection-guided densification strategy to enhance specular pattern learning
- State-of-the-art novel view synthesis performance across synthetic and real reflective datasets
Why it matters
Enables high-fidelity 3D reconstruction and rendering for downstream applications like robotic grasping, autonomous driving, and AR/VR that rely on accurate specular and geometry cues.
Abstract
Gaussian Splatting has significantly improved the quality of novel view synthesis with explicit Gaussian representation. However, we observed that existing 3D Gaussian Splatting methods (3DGS) often suffer from surface collapse issues on reflective regions, and thus produce inferior geometry and low-quality specular. In this work, we propose a physically- based deferred rendering framework, named Reflection-aware Gaussian Splatting (RGS), that can accurately model specular regions and improve novel view synthesis performance. Specifi- cally, we found that a powerful 3D foundation model can provide a strong 3D geometric prior to foster correct geometric modeling. Based on this, we propose a cross-view shape consistency regularization to regularize the geometry surface with the large model prior and cross-view constraints. In this manner, our RGS can produce smoother geometric surfaces on reflective regions while reducing geometric hollows. To further improve rendering results on reflective regions, we present a reflection- aware densification strategy that is designed to capture specular variations across various views. With this strategy, our RGS is able to render novel views of objects in higher quality. Extensive experiments demonstrate our method consistently renders high- quality reflective objects, achieving state-of-the-art performance. Project Page: https://xiaobiaodu.github.io/reflectivegs/