Learning Perception-Aware Agile Flight in Cluttered Environments

Recently, neural control policies have outper- formed existing model-based planning-and-control methods for autonomously navigating quadrotors through cluttered envi- ronments in minimum time. However, they are not perception aware, a crucial requirement in vision-based navigation due to the camera’s limited field of view and the underactu- ated nature of a quadrotor. We propose a learning-based system that achieves perception-aware, agile flight in clut- tered environments. Our method combines imitation learning with reinforcement learning (RL) by leveraging a privileged learning-by-cheating framework. Using RL, we first train a perception-aware teacher policy with full-state information to fly in minimum time through cluttered environments. Then, we use imitation learning to distill its knowledge into a vision- based student policy that only perceives the environment via a camera. Our approach tightly couples perception and control, showing a significant advantage in computation speed (10× faster) and success rate. We demonstrate the closed-loop control performance using hardware-in-the-loop simulation. Video: https://youtu.be/9q059CFGcVA ∗These two authors contributed equally. 1Y. Song, K. Shi, and D. Scara- muzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland (http://rpg.ifi.uzh.ch). 2R. Penicka is with the Multi-robot Systems Group, Czech Technical University in Prague, Czech Republic. This work was supported by the Swiss National Science Foundation (SNSF) through the National Centre of Competence in Research (NCCR) Robotics, the Czech Science Foundation (GA ˇCR) under research projects No. 23-06162M, the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No. 871479 (AERIAL-CORE), and the European Research Council (ERC) under grant agreement No. 864042 (AGILEFLIGHT).