Targetless LiDAR-Camera Calibration with Neural Gaussian Splatting
Haebeom Jung, Namtae Kim, Jungwoo Kim, Jaesik Park
AI summary
Problem
Traditional calibration relies on impractical physical targets or degrades over time, while existing targetless methods struggle with LiDAR sparsity, high computational costs, and poor generalization across diverse scenes.
Approach
The method jointly optimizes camera extrinsics and a neural Gaussian scene representation using frozen LiDAR points as structural anchors and learnable auxiliary Gaussians for local refinement, guided by photometric and geometric losses.
Key results
- 100% calibration success rate with minimal rotation and translation errors on KITTI-360
- Training time reduced to ~0.18 hours, significantly faster than baseline methods
- Superior novel view synthesis quality across KITTI-360, WAYMO, and FAST-LIVO2 datasets
- Robust generalization to diverse motion patterns and handheld solid-state LiDAR setups
Why it matters
Provides a scalable, target-free solution for maintaining accurate multi-sensor alignment in autonomous driving and robotics applications.
Abstract
Accurate LiDAR-camera calibration is crucial for multi-sensor systems. However, traditional methods often rely on physical targets, which are impractical for real-world deployment. Moreover, even carefully calibrated extrinsics can degrade over time due to sensor drift or external distur- bances, necessitating periodic recalibration. To address these challenges, we present a Targetless LiDAR–Camera Calibration (TLC-Calib) that jointly optimizes sensor poses with a neural Gaussian–based scene representation. Reliable LiDAR points are frozen as anchor Gaussians to preserve global structure, while auxiliary Gaussians prevent local overfitting under noisy initialization. Our fully differentiable pipeline with photometric and geometric regularization achieves robust and generalizable calibration, consistently outperforming existing targetless meth- ods on the KITTI-360, WAYMO, and FAST-LIVO2 datasets. In addition, it yields more consistent Novel View Synthesis results, reflecting improved extrinsic alignment. The project page is available at: https://www.haebeom.com/tlc-calib-site/.