Tube-GAN: A Novel Virtual Tube Generation Method for Unmanned Aerial Swarms Based on Generative Adversarial Network

Virtual tube is a two-dimensional or three-dimensional strip or tubular area similar to RSFC (Relative Safe Flight Corridor), which can provide smooth, feasible, and safe space for UAV swarm in environments with dense obstacles. In order to address the problem that current virtual tube planning methods are mainly based on complex heuristic algorithm with consuming time complexity, we modify the model architecture by introducing generative adversarial network (GAN), and propose a Tube-GAN model. Tube-GAN takes the key point prompt image and obstacle environment image as inputs, and outputs the image of the virtual tube, which transforms the optimization problem into an image generation problem, leveraging the performance of computational efficiency for the construction of virtual tube. The experimental results demonstrate that the proposed Tube-GAN model can quickly generate virtual tube in random environments (less than 25ms), providing a new direction for the construction of virtual tube in real-time. INTRODUCTION Unmanned aerial swarm systems have attracted more and more attentions from industry and military applications because of the advantages in autonomy, collaboration, robustness and scalability. The unmanned swarms can perform more complex and diverse tasks with higher efficiency and lower cost comparing with using single robotic machine in many fields, especially under harsh environmental conditions [1],[2]. However, the rapid and collision-free traversal for unmanned aerial swarm systems in complex environments such as forests, valleys, and cities is a challenging problem that has not been well solved. Reference [3] uses a distributed architecture to generate trajectory for each UAV following some priority order, and broadcasts the generated trajectory to other UAVs as obstacle information for trajectory planning. Such methods can achieve high-speed control for each UAV, but they can easily lead to deadlock problems, especially when the number of UAV individuals is large. Reference [4] designed a position and velocity consistency control law between the leader UAV and the follower UAVs to maintain the formation of the aerial swarm where artificial potential field method was introduced. Such formation control-based method can avoid deadlocks between UAVs, but the traffic efficiency is poor. In addition, by mimicking bird swarm behaviors, Reynolds et al. [5] Shixun Zhai, Kaige Zhang (corresponding author), Bo Nan and Yanwen Sun are with the North Automatic Control Institute, Taiyuan, Shanxi 030000 China (e-mail: zhaishixun@126.com; zkgusu@gmail.com). Qianyi Fu is with School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310014 China (e-mail: scfq@leeds.ac.uk). proposed the Boyd model, followed by a number of biomimicry swarm motion models, such as Vicsek model, Couzin model, and Cucker-Smale model. These methods achieved some success in obstacle avoidance with large-scale unmanned aerial swarms, but the traffic efficiency problem is still not resolved. Fig. 1 Virtual tube in three-dimensional spaces Virtual tube was proposed to address the contradiction between the increasing use of UAVs and limited airspace [6]. For vehicle motion planning, an Air Traffic Management System (ATM) is constructed to confine UAVs within planned virtual tube and to ensure that newly added UAVs could avoid collisions with other existing UAVs. Combining trajectory planning and swarm control algorithms, reference [7] proposed a swarm flight control approach with virtual tube. The virtual tube is constructed from the traditional trajectory planning prospect, and swarm control methods such as the flocking model are used to achieve collaborative control of the unmanned aerial swarms within the virtual tube. However, the construction of virtual tube is based on traditional heuristic algorithm with strict mathematical constraints, and that is time consuming and cannot meet the requirements of UAV swarm systems. In this work, we propose a novel virtual tube planning method, named Tube-GAN, which transforms the virtual tube construction problem into an image generation problem by introducing the generative adversarial network technology. Tube-GAN takes the key point prompt image and obstacle environment image as inputs, outputs the environmental image embedded with well-planned virtual tubes, resulting in a significant performance improvement of computational efficiency, and providing a new solution for swarm trajectory planning based on virtual tube generation. The main contributions of this work are: 1) Propose a novel virtual tube generation method based on GAN, which for the first time transforms the virtual tube construction problem into an image generation problem; 2) Design, implement, and evaluate the Tube-GAN model for swarm control in the environment with dense obstacles; 3) Validate the effectiveness and efficiency of the proposed method by simulating the full process of UAV Tube-GAN: A Novel Virtual Tube Generation Method for Unmanned Aerial Swarms Based on Generative Adversarial Network Shixun Zhai, Kaige Zhang, Bo Nan, Yanwen Sun, and Qianyi Fu 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) October 14-18, 2024. Abu Dhabi, UAE 979-8-3503-7769-9/24/$31.00 ©2024 IEEE 11818