Ultra-Low Power Deep Learning-Based Monocular Relative Localization Onboard Nano-Quadrotors

Precise relative localization is a crucial functional block for swarm robotics. This work presents a novel au- tonomous end-to-end system that addresses the monocular relative localization, through deep neural networks (DNNs), of two peer nano-drones, i.e., sub-40 g of weight and sub-100 mW processing power. To cope with the ultra-constrained nano- drone platform, we propose a vertically-integrated framework, from the dataset collection to the final in-field deployment, including dataset augmentation, quantization, and system op- timizations. Experimental results show that our DNN can precisely localize a 10 cm-size target nano-drone by employing only low-resolution monochrome images, up to ∼2 m distance. On a disjoint testing dataset our model yields a mean R2 score of 0.42 and a root mean square error of 18 cm, which results in a mean in-field prediction error of 15 cm and in a closed-loop control error of 17 cm, over a ∼60 s-flight test. Ultimately, the proposed system improves the State-of-the- Art by showing long-endurance tracking performance (up to 2 min continuous tracking), generalization capabilities being deployed in a never-seen-before environment, and requiring a minimal power consumption of 95 mW for an onboard real-time inference-rate of 48 Hz. SUPPLEMENTARY VIDEO MATERIAL In-field tests: https://youtu.be/pUGL1qu3Z1k.