JuggleRL: Mastering Ball Juggling with a Quadrotor Via Deep Reinforcement Learning
Shilong Ji, Yinuo Chen, Chuqi Wang, Jiayu Chen, Ruize Zhang, Feng Gao, Wenhao Tang, Shu'ang Yu, Sirui Xiang, Xinlei Chen, Chao Yu, Yu Wang
AI summary
Problem
Traditional model-based approaches for aerial ball juggling are brittle due to nonlinear dynamics, prediction error accumulation, and rigid trajectory constraints that fail under real-world uncertainty.
Approach
JuggleRL trains a reinforcement learning policy in a large-scale, domain-randomized simulation with calibrated dynamics and reward shaping, then deploys it zero-shot on real hardware using a low-latency perception module.
Key results
- Achieves up to 462 consecutive hits (average 311) in real-world trials
- Outperforms model-based baseline by over 20x (baseline average 3.1 hits)
- Generalizes to unseen ball weights, juggling a 5g ball with 145.9 average hits
- Enables zero-shot sim-to-real transfer with reduced communication latency
Why it matters
Demonstrates that reinforcement learning can reliably handle dynamic, contact-rich aerial manipulation, advancing robust real-time control for autonomous drones.
Abstract
Aerial robots interacting with objects must perform precise, contact-rich maneuvers under uncertainty. In this paper, we study the problem of aerial ball juggling using a quadrotor equipped with a racket, a task that demands accurate timing, stable control, and continuous adaptation. We propose JuggleRL, the first reinforcement learning-based system for aerial juggling. It learns closed-loop policies in large-scale simulation using systematic calibration of quadrotor and ball dynamics to reduce the sim-to-real gap. The training incorporates reward shaping to encourage racket-centered hits and sustained juggling, as well as domain randomization over ball position and coefficient of restitution to enhance robustness and transferability. The learned policy outputs mid-level commands executed by a low-level controller and is deployed zero-shot on real hardware, where an enhanced perception module with a lightweight communication protocol reduces delays in high-frequency state estimation and ensures real-time control. Experiments show that JuggleRL achieves an average of 311 hits over 10 consecutive trials in the real world, with a maximum of 462 hits observed, far exceeding a model-based baseline that reaches at most 14 hits with an average of 3.1. Moreover, the policy generalizes to unseen conditions, successfully juggling a lighter 5 g ball with an average of 145.9 hits. This work demonstrates that reinforcement learning can empower aerial robots with robust and stable control in dynamic interaction tasks. The project page is at https://jugglerl.github.io/.