SplatSDF: Boosting SDF-NeRF Via Architecture-Level Fusion with Gaussian Splats
Runfa Li, Daniel George, Keito Suzuki, Bang Du, Ki Myung Brian Lee, Nikolay Atanasov, Truong Nguyen
AI summary
Problem
SDF-NeRF models provide precise geometric and photometric rendering but suffer from slow training convergence and difficulty capturing complex geometry, limiting their use in practical robotic systems.
Approach
The method pre-trains a 3D Gaussian splat model and injects its neural embeddings directly into the SDF-NeRF during training using a sparse surface-fusion strategy, removing the need for Gaussians at inference time.
Key results
- Achieves >3× faster convergence to target geometric accuracy than Neuralangelo
- Outperforms state-of-the-art SDF-NeRF methods in Chamfer distance and PSNR
- Eliminates spurious surface artifacts via sparse surface-only embedding fusion
- Accelerates gradient and Hessian computations by 3× through voxel hashing
Why it matters
Provides a faster, more robust pathway for deploying high-fidelity 3D environment representations in real-time robotic perception and manipulation.
Abstract
Signed distance-radiance field (SDF-NeRF) is a promising environment representation that offers both photo- realistic rendering and geometric reasoning such as proximity queries for collision avoidance. However, the slow training speed and convergence of SDF-NeRF hinder their use in practical robotic systems. We propose SplatSDF, a novel SDF-NeRF architecture that accelerates convergence using 3D Gaussian splats (3DGS), which can be quickly pre-trained. Unlike prior approaches that introduce a consistency loss between separate 3DGS and SDF-NeRF models, SplatSDF directly fuses 3DGS at an architectural level by consuming it as an input to SDF- NeRF during training. This is achieved using a novel sparse 3DGS fusion strategy that injects neural embeddings of 3DGS into SDF-NeRF around the object surface, while also permitting inference without 3DGS for minimal operation. Experimental results show SplatSDF achieves 3× faster convergence to the same geometric accuracy than the best baseline, and outper- forms state-of-the-art SDF-NeRF methods in terms of chamfer distance and peak signal to noise ratio, unlike consistency loss-based approaches that in fact provide limited gains. We also present computational techniques for accelerating gradient and Hessian steps by 3×. We expect these improvements will contribute to deploying SDF-NeRF on practical systems.