Learning to Drive by Imitating Surrounding Vehicles
Yasin Sonmez, Hanna Krasowski, Murat Arcak
AI summary
Problem
Imitation learning for autonomous driving often suffers from distribution shift, causal confusion, and an over-reliance on routine ego-vehicle data, while existing augmentation methods either ignore surrounding traffic or rely on unrealistic simulations.
Approach
The authors introduce a filtering and sampling strategy that prioritizes diverse, dynamically complex trajectories from nearby vehicles based on heading deviations and comfort metrics, then transforms them into the ego-vehicle's reference frame to enrich the training dataset.
Key results
- Reduced collision rates and improved safety metrics on the nuPlan benchmark
- Matching or exceeding full-dataset performance using only 10% of the training data
- Demonstrated that naive random vehicle selection degrades policy performance
- Validated the augmentation framework on large-scale real-world object-based planning data
Why it matters
Offers a scalable, simulation-free data curation method that enables safer autonomous vehicle training with significantly less real-world data.
Abstract
Imitation learning is a promising approach for training autonomous vehicles (AV) to navigate complex traffic environments by mimicking expert driver behaviors. While ex- isting imitation learning frameworks focus on leveraging expert demonstrations, they often overlook the potential of additional complex driving data from surrounding traffic participants. In this paper, we study a data augmentation strategy that leverages the observed trajectories of nearby vehicles, captured by the AV’s sensors, as additional demonstrations. We introduce a simple vehicle-selection sampling and filtering strategy that pri- oritizes informative and diverse driving behaviors, contributing to a richer dataset for training. We evaluate this idea with a representative learning-based planner on a large real-world dataset and demonstrate improved performance in complex driving scenarios. Specifically, the approach reduces collision rates and improves safety metrics compared to the baseline. Notably, even when using only 10 percent of the original dataset, the method matches or exceeds the performance of the full dataset. Through ablations, we analyze selection criteria and show that naive random selection can degrade performance. Our findings highlight the value of leveraging diverse real-world trajectory data in imitation learning and provide insights into data augmentation strategies for autonomous driving.