DRUM: Diffusion-Based Raydrop-Aware Unpaired Mapping for Sim2Real LiDAR Segmentation
Tomoya Miyawaki, Kazuto Nakashima, Yumi Iwashita, Ryo Kurazume
AI summary
Problem
Manual annotation of LiDAR point clouds is prohibitively expensive, and models trained on synthetic data fail to generalize to real-world scenes due to unaddressed domain gaps in reflectance intensity and raydrop noise.
Approach
The method formulates Sim2Real translation as posterior sampling using a diffusion model pre-trained on unlabeled real data, guided by a novel raydrop-aware masking strategy that preserves realistic sensor noise while maintaining geometric consistency with synthetic inputs.
Key results
- Novel diffusion-based Sim2Real translation framework for LiDAR point clouds
- Raydrop-aware masked guidance mechanism that prevents over-inpainting and stabilizes generation
- Superior sample fidelity compared to existing domain adaptation baselines
- Consistent improvements in Sim2Real semantic segmentation accuracy across multiple data representations
Why it matters
Provides a scalable, annotation-free pipeline for training robust LiDAR perception models, accelerating deployment in autonomous robotics and driving.
Abstract
LiDAR-based semantic segmentation is a key com- ponent for autonomous mobile robots, yet large-scale annotation of LiDAR point clouds is prohibitively expensive and time- consuming. Although simulators can provide labeled synthetic data, models trained on synthetic data often underperform on real-world data due to a data-level domain gap. To address this issue, we propose DRUM, a novel Sim2Real translation framework. We leverage a diffusion model pre-trained on unlabeled real-world data as a generative prior and translate synthetic data by reproducing two key measurement charac- teristics: reflectance intensity and raydrop noise. To improve sample fidelity, we introduce a raydrop-aware masked guidance mechanism that selectively enforces consistency with the input synthetic data while preserving realistic raydrop noise induced by the diffusion prior. Experimental results demonstrate that DRUM consistently improves Sim2Real performance across multiple representations of LiDAR data. The project page is available at https://miya-tomoya.github.io/drum.