Performance-Guided Refinement for Visual Aerial Navigation Using Editable Gaussian Splatting in FalconGym 2.0
Yan Miao, Ege Yuceel, Georgios Fainekos, Bardh Hoxha, Hideki Okamoto, Sayan Mitra
AI summary
Problem
State-of-the-art visual policies for aerial navigation overfit to specific training tracks and fail to generalize to new geometries, requiring costly per-track data collection and retraining.
Approach
The authors introduce FalconGym 2.0, a photorealistic simulator with a programmatic Edit API for rapid track generation, and develop a Performance-Guided Refinement algorithm that iteratively targets challenging tracks to train a robust, modular visual policy.
Key results
- FalconGym 2.0 enables millisecond-scale generation of diverse photorealistic tracks via an editable GSplat API
- The PGR algorithm trains a single visual policy that achieves 100% success across three unseen simulation tracks without per-track retraining
- The policy maintains higher robustness under gate-pose perturbations compared to state-of-the-art baselines
- Zero-shot sim-to-real transfer to quadrotor hardware achieves a 98.6% success rate across multiple tracks
Why it matters
It eliminates the need for per-track data collection and retraining, enabling robust, generalizable visual navigation for aerial robots in dynamic real-world environments.
Abstract
Visual policy design is crucial for aerial naviga- tion. However, state-of-the-art visual policies often overfit to a single track and their performance degrades when track geom- etry changes. We develop FalconGym 2.0, a photorealistic sim- ulation framework built on Gaussian Splatting (GSplat) with an Edit API that programmatically generates diverse static and dynamic tracks in milliseconds. Leveraging FalconGym 2.0’s editability, we propose a Performance-Guided Refinement (PGR) algorithm, which concentrates visual-policy training on chal- lenging tracks while iteratively improving performance. Across two case studies (fixed-wing UAVs and quadrotors) with distinct dynamics and environments, we show that a single visual policy trained with PGR in FalconGym 2.0 outperforms state-of-the- art baselines in generalization and robustness: it generalizes to three unseen tracks with 100% success without per-track retraining and maintains higher success rates under gate- pose perturbations. Finally, we demonstrate zero-shot sim-to- real transfer of the PGR-trained visual policy to quadrotor hardware, achieving a 98.6% success rate (69/70 gates) over 30 trials across two three-gate tracks and one moving-gate track.