Autonomous Drone-Ground Robot Alignment through Ground Robot Visual Servo Control with Drone Detection and Tilt Correction
Sean Clark Dominguez, Jeanette Pao, Immanuel Paradela, John Mel Bolaybolay, Earl Ryan Aleluya, Francis Jann Alagon, Sherwin Guirnaldo, Carl John Salaan, Kazunori Ohno, Yoshito Okada, Argel Bandala, Abu Ubaidah Shamsudin
AI summary
Problem
Retrieving ground robots from hazardous environments is risky and traditionally relies on error-prone GPS or unstable aerial visual servoing, which struggle with drone drift, wind gusts, and terrain-induced positioning errors.
Approach
The system uses the ground robot’s upward-facing camera to detect a hovering drone via YOLOv8 instance segmentation, calculates its relative distance and heading, and applies a tilt correction algorithm to maintain alignment on uneven surfaces.
Key results
- Trained YOLOv8 instance segmentation model for drone detection across varying weather conditions
- Developed a tilt correction algorithm to compensate for ground robot platform inclination on uneven surfaces
- Achieved a maximum alignment error of 20.3 cm in outdoor field tests, meeting the retrieval threshold
- Validated functional ground robot visual servo control across multiple rough terrain conditions
Why it matters
Enables safer, autonomous retrieval of exploration robots in hazardous environments by eliminating reliance on error-prone GPS and reducing risks associated with aerial visual servoing.
Abstract
Retrieving ground robots from dangerous environ- ments after their operation is a challenging task that poses risks for the personnel. Some researchers often employ drones for retrieval, which makes operations safer. However, this set-up requires an ac- curate method that guarantees drone and ground robot alignment due to inaccuracies in standard GPS devices, drone drifts, and wind gusts. Hence, this research article introduces simultaneous object detection and tilt correction as part of visual servoing to achieve precisedrone-roveralignment.DronedetectionusingYOLOv8and a tilt correction algorithm was integrated for the proposed visual servo of the ground robot. The study collected 3024 images as a data set for drone detection. The experimental results show that the trained instance segmentation model detected and captured drone objects. The study conducted an initial test for visual servo control of the ground robot in various surface terrains, resulting in a maximum alignment error on rough surfaces. Furthermore, the study conducted drone-ground robot alignment real test in an outdoor field setting. The alignment between the drone and the ground robot produced a maximum alignment error of 20.3 cm, below the threshold error. The open field experiments verified the effectiveness of the ground robot’s visual servo control with an actual drone operation. Received 23 January 2025; accepted 21 April 2025. Date of publication 12 May 2025; date of current version 26 May 2025. This article was recommended for publication by Associate Editor E. Malis and Editor P. Vasseur upon eval- uation of the reviewers’ comments. This work was supported in part by the PhilippineCouncilforIndustry,Energy,andEmergingTechnologyResearchand Development (PCIEERD) and in part by the Philippine Institute of Volcanology and Seismology (PHIVOLCS). (Corresponding author: Carl John Salaan.) Sean Clark Dominguez, Jeanette Pao, Immanuel Paradela, John Mel Bolaybo- lay, Earl Ryan Aleluya, Francis Jann Alagon, Sherwin Guirnaldo, and Carl John Salaan are with the College of Engineering, Mindanao State University–Iligan Institute of Technology, Iligan City 9200, Philippines (e-mail: seanclark. dominguez@g.msuiit.edu.ph; jeanette.pao@g.msuiit.edu.ph; immanuel. paradela@g.msuiit.edu.ph; johnmel.bolaybolay@g.msuiit.edu.ph; earlryan. aleluya@g.msuiit.edu.ph; francisjann.alagon@g.msuiit.edu.ph; sherwin. guirnaldo@g.msuiit.edu.ph; carljohn.salaan@g.msuiit.edu.ph). Kazunori Ohno and Yoshito Okada are with the Graduate School of Information Sciences, Tohoku University, Sendai 980-8577, Japan (e-mail: kazunori@rm.is.tohoku.ac.jp; yoshito@rm.is.tohoku.ac.jp). Argel Bandala is with the Gokongwei College of Engineering, De La Salle University, Manila 1004, Philippines (e-mail: argel.bandala@dlsu.edu.ph). Abu Ubaidah Bin Shamsudin is with the Faculty of Electric and Electronic Engineering, Universiti Tun Hussein Onn Malaysia, Batu Pahat 86400, Malaysia (e-mail: ubaidah@uthm.edu.my). This article has supplementary downloadable material available at https://doi.org/10.1109/LRA.2025.3569125, provided by the authors. Digital Object Identifier 10.1109/LRA.2025.3569125