LiHi-GS: LiDAR-Supervised Gaussian Splatting for Highway Driving Scene Reconstruction
Pou-Chun Kung, Xianling Zhang, Katherine Skinner, Nikita Jaipuria
AI summary
Problem
Current Gaussian Splatting methods for autonomous driving are limited to urban environments and superficially use LiDAR data, failing to handle highway-specific challenges like sparse viewpoints, monotone backgrounds, and long-range objects while lacking realistic LiDAR synthesis capabilities.
Approach
LiHi-GS integrates a differentiable LiDAR sensor model directly into Gaussian Splatting training to enable precise depth supervision and novel-view LiDAR rendering, alongside decoupled pose optimization to correct camera-LiDAR misalignments.
Key results
- First comprehensive evaluation of Gaussian Splatting on long-range highway scenes
- Introduction of a differentiable LiDAR rendering model for direct supervision and synthesis
- State-of-the-art performance in image and LiDAR novel view rendering
- Demonstrated critical role of LiDAR supervision in improving geometry learning for feature-sparse environments
Why it matters
It enables cost-effective generation of photorealistic, sensor-accurate synthetic data for training and testing autonomous driving systems in critical, underrepresented highway scenarios.
Abstract
Photorealistic 3D scene reconstruction plays an important role in autonomous driving, enabling the generation of novel data from existing datasets to simulate safety-critical scenarios and expand training data without additional acquisition costs. Gaussian Splatting (GS) facilitates real-time, photorealistic rendering with an explicit 3D Gaussian representation of the scene, providing faster processing and more intuitive scene editing than the implicit Neural Radiance Fields (NeRFs). While extensive GS research has yielded promising advancements in au- tonomous driving applications, they overlook two critical aspects. First, existing methods mainly focus on low-speed and feature- rich urban scenes and ignore the fact that highway scenarios play a significant role in autonomous driving. Second, while LiDARs are commonplace in autonomous driving platforms, existing methods learn primarily from images and use LiDAR only for initial estimates or without precise sensor modeling, thus missing out on leveraging the rich depth information LiDAR offers and limiting the ability to synthesize LiDAR data. In this paper, we propose a novel GS method for dynamic scene synthesis and editing with improved scene reconstruction through LiDAR supervision and support for LiDAR rendering. Unlike prior works that are tested mostly on urban datasets, to the best of our knowledge, we are the first to focus on the more challenging and highly relevant highway scenes for autonomous driving, which feature sparse sensor views and monotone backgrounds. A project page is available at https://umautobots.github.io/lihi gs.