Nighttime Autonomous Driving Scene Reconstruction with Physically-Based Gaussian Splatting
Tae-Kyeong Kim, Xingxin Chen, Guile Wu, Chengjie Huang, Dongfeng Bai, Bingbing Liu
AI summary
Problem
Existing 3D Gaussian Splatting and NeRF methods for autonomous driving fail to accurately model low-light conditions, suffering from performance degradation due to complex lighting and dynamic scene requirements.
Approach
The method decomposes scene lighting into diffuse and specular components, using a global spherical harmonic illumination module for diffuse light and anisotropic spherical Gaussians with BRDF constraints for specular highlights, all optimized within a composite 3DGS scene graph.
Key results
- Surpasses state-of-the-art baselines on Waymo and nuScenes datasets
- Accurately captures sharp specular highlights and global illumination without environment maps
- Maintains real-time rendering speeds for dynamic nighttime scenes
- Reduces perceptual error while improving structural fidelity across diverse low-light scenarios
Why it matters
Provides a robust, real-time simulation tool for testing autonomous driving systems in safety-critical nighttime environments.
Abstract
This paper focuses on scene reconstruction under nighttime conditions in autonomous driving simulation. Re- cent methods based on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) have achieved photorealistic modeling in autonomous driving scene reconstruction, but they primarily focus on normal-light conditions. Low-light driving scenes are more challenging to model due to their complex lighting and appearance conditions, which often causes performance degradation of existing methods. To address this problem, this work presents a novel approach that integrates physically based rendering into 3DGS to enhance nighttime scene reconstruction for autonomous driving. Specifically, our approach integrates physically based rendering into composite scene Gaussian representations and jointly optimizes Bidirec- tional Reflectance Distribution Function (BRDF) based material properties. We explicitly model diffuse components through a global illumination module and specular components by anisotropic spherical Gaussians. As a result, our approach improves reconstruction quality for outdoor nighttime driving scenes, while maintaining real-time rendering. Extensive exper- iments across diverse nighttime scenarios on two real-world autonomous driving datasets, including nuScenes and Waymo, demonstrate that our approach outperforms the state-of-the-art methods both quantitatively and qualitatively.