Preserving Relative Localization of FoV-Limited Drone Swarm Via Active Mutual Observation

Relative state estimation is crucial for vision-based swarms to estimate and compensate for the unavoidable drift of visual odometry. For autonomous drones equipped with the most compact sensor setting — a stereo camera that provides a limited field of view (FoV), the demand for mutual observation for relative state estimation conflicts with the demand for environment observation. To balance the two demands for FoV- limited swarms by acquiring mutual observations with a safety guarantee, this paper proposes an active localization correction system, which plans camera orientations via a yaw planner during the flight. The yaw planner manages the contradiction by calculating suitable timing and yaw angle commands based on the evaluation of localization uncertainty estimated by the Kalman Filter. Simulation validates the scalability of our algorithm. In real-world experiments, we reduce positioning drift by up to 65% and managed to maintain a given formation in both indoor and outdoor GPS-denied flight, from which the accuracy, efficiency, and robustness of the proposed system are verified. I. INTRODUTION Micro vision-based aerial swarms have become popular for their low cost, agility, and independence of bulky external sensors. For some swarm missions like formation flight [1], coordinated object handling [2], and collaborative mapping and exploration [3], accurate alignment of the reference frames maintained by each agent is a basic requirement. However, due to the unavoidable drift of vision-based lo- calization, the alignment breaks during mission execution, requiring continuous relative state estimation among vehicles for frame re-alignment. To conduct continuous relative state estimation in vision- based swarms, mutual observation is adopted for its environment-independence. In existing methods, mutual ob- servation is achieved through drone detection via onboard cameras. Then relative position can be derived from the drone detection [4]–[6]. Vision-based mutual observation requires drones to be captured by cameras of others, hence some works add extra sensors like multiple fisheye cameras with an omnidirectional sensing range to capture all nearby drones †Equal contribution. 1Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China. 2Huzhou Institute, Zhejiang University, Huzhou 313000, China. 3The Hong Kong University of Science and Technology (GZ). ∗Corresponding authors: Xin Zhou and Fei Gao. This work was supported by the National Key R&D Program of China under grant no. 2023YFB4706600 and the National Natural Science Foun- dation of China under grant no. 62322314. E-mail:{iszhouxin, fgaoaa}@zju.edu.cn Fig. 1: Leverage active mutual observation for localization correction in the field experiment. (A) The drones are disorganized due to the drift of the VIO. (B) The mutual observation tasks are assigned, drone 0 observes drone 1, and drone 1 observes drone 2 (white arrows). The yaw rotation (black arrows) can be seen more clearly in the close- up views. (C) After mutual observation, relative localization is corrected. Drones fly in the predefined line formation. (D) Drones conduct environment observation and deform the formation to avoid the tree obstacles. The red curves are the approximate flight paths. [7]. However, for micro aerial vehicles, the limited payload capacity makes it tough to install additional sensors. This pa- per aims to preserve relative localization with a minimum set of sensors widely applied in autonomous drone navigation: a field-of-view-limited (FoV-limited) stereo camera with an IMU. For FoV-limited swarms, when estimating the relative localization with the only camera, there exists a contradiction between two observations: 1) the environment observation for obstacle avoidance; 2) the mutual observation for relative state estimation (Fig. 2). To guarantee flight safety, the drone 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) October 14-18, 2024. Abu Dhabi, UAE 979-8-3503-7769-9/24/$31.00 ©2024 IEEE 10422