Conditional Flow-VAE for Safety-Critical Traffic Scenario Generation
Zimu Gong, Brian Zhaoning Zhang, Chris Zhang, Kelvin Wong, Raquel Urtasun
AI summary
Problem
Safety-critical driving events are rare in real data, and current simulation methods either scale poorly due to manual curation or produce unrealistic adversarial behaviors, hindering robust autonomous vehicle testing.
Approach
The method trains a conditional VAE on mixed real and synthetic data, then uses a flow matching transformer to learn a latent-space transport map that steers nominal scenarios toward safety-critical distributions.
Key results
- Highest near-miss generation rate among evaluated baselines
- High distributional similarity to real-world driving statistics
- Controllable scenario difficulty via heuristic maneuver labels
- Superior realism and diversity compared to VAE and STRIVE baselines
Why it matters
Enables scalable, realistic stress-testing for autonomous vehicles, accelerating safe deployment and benchmarking.
Abstract
Safety-critical scenarios are essential for the devel- opment of autonomous vehicles (AVs) but are rare in real-world driving data. While simulation offers a way to generate such scenarios, manually designed test cases lack scalability, and adversarial optimization often produces unrealistic behaviors. In this work, we introduce a conditional latent flow matching approach for scalable and realistic safety-critical scenario gen- eration. Our method uses distribution matching to transform nominal scenes into safety-critical rollouts. Furthermore, we demonstrate that incorporating both simulation and real-world data enables our framework to efficiently generate diverse, data-driven scenarios. Experimental results highlight that our approach is able to more consistently and realistically generate novel safety-critical scenarios, making it a valuable tool for training and benchmarking AV systems.