Multi-View Control for Robust 3D Gaussian Splatting
YuNong Mao, Zhibin Zhang, yufu shi
AI summary
Problem
3D Gaussian Splatting performance degrades with poor initial point clouds because erroneous Gaussian points accumulate and cause overfitting to training views, distorting novel view synthesis.
Approach
The method introduces a Dual-view Point Correction mechanism to identify and prune erroneous Gaussians using multi-view geometry, alongside a multi-view joint training strategy that balances gradients across perspectives to prevent overfitting.
Key results
- Achieves state-of-the-art PSNR, SSIM, and LPIPS across Mip-NeRF 360, Tanks&Temples, and Deep Blending datasets
- Eliminates dependency on high-quality SfM initialization by matching performance with randomly initialized point clouds
- Dual-view point correction effectively reduces rendering errors and eliminates image distortion in novel views
- Multi-view joint training improves reconstruction of sharp edges and fine details by balancing cross-view gradients
Why it matters
Enables high-fidelity 3D reconstruction and real-time rendering in real-world scenarios where accurate camera poses or initial point clouds are unavailable.
Abstract
3D Gaussian Splatting (3DGS) has recently demonstrated impressive capabilities in real-time novel view synthesis. However, the performance of 3DGS tends to degrade significantly when the quality of the initial point cloud is poor. Specifically, the lack of an effective pruning strategy to thoroughly eliminate suboptimal points (defined as erroneous points in this paper). The excessive accumulation of these erro- neous points leads to overfitting in specific viewpoints, thereby affecting the visual appearance and geometric accuracy in novel view synthesis. To address these challenges, we propose a novel 3DGS optimization method named MVC-GS, which introduces two key innovative contributions. First, based on multi-view geometric constraints, we use image rendering errors as a guiding criterion for optimization. By performing point cali- bration in the target region, we effectively mitigate the impact of erroneous Gaussian points. Subsequently, we introduce a multi-view Gaussian attribute optimization method that further enhances the precision of 3D Gaussian attributes representation, while avoiding overfitting to the training views. We conducted comprehensive visualization analysis across multiple scenes in various datasets. Extensive experiments on public datasets show that the proposed method achieves state-of-the-art performance across diverse scenes.