Flying Together: Human-Guided Immersive Shared Control for Aerial Robot Teams in Unknown Environments
Lou De Bel-Air, Luca Morando, Ruitao Chen, Keru Wang, Benjamin Jarvis, Charbel Toumieh, Yang Zhou, KEN PERLIN, Dario Floreano, Giuseppe Loianno
AI summary
Problem
Autonomous multi-robot teams struggle to adapt to unforeseen conditions and incorporate real-time human objectives in unstructured environments, while existing teleoperation methods lack scalable, immersive interfaces for intuitive swarm guidance.
Approach
The authors develop a VR-based shared control framework that combines a novel motion-primitive planner with a real-time user-alignment term and an admittance controller, allowing operators to intuitively guide drone teams via a WebXR interface while ensuring dynamically feasible, collision-free trajectories.
Key results
- Novel motion-primitive planner with real-time user-alignment term
- Variable admittance controller for dynamically feasible team migration
- Open-source WebXR interface enabling low-latency immersive control
- Experimental validation showing improved obstacle avoidance, maintained formation, and reduced operator effort
Why it matters
Provides a scalable, accessible framework for human-in-the-loop swarm control, critical for search and rescue and infrastructure inspection where autonomous systems alone are insufficient.
Abstract
While autonomous multi-robots can achieve safe and coordinated navigation, they often struggle to adapt to unforeseen conditions and to capture operator-driven objectives in unstructured environments. We present a Virtual Reality (VR)-based shared control framework for teams of drones operating in constrained and unknown environments, enabling real-time, user-guided exploration. At the core of our approach is a novel, user-guided motion-primitive-based planner that computes continuous, collision-free trajectories while contin- uously integrating operator input. This planner is coupled with an admittance controller, allowing the operator to flexibly influence team behavior and guide drones toward regions of interest that autonomous planners may overlook. The system supports mixed-reality operations with both physical and sim- ulated drones, and implements a bilateral VR-based interface, allowing the operator to guide the robot team via migration points while receiving immediate visual feedback of the team state. Experimental results show that shared control improves obstacle avoidance, maintains inter-agent spacing, and reduces operator effort, demonstrating the feasibility and advantages of immersive, human-in-the-loop multi-robot navigation.