Reinforcement Learning for Robust Athletic Intelligence: Lessons from the 2nd �AI Olympics with RealAIGym� Competition
Felix Wiebe, Niccolò Turcato, Alberto Dalla Libera, Jean Seong Bjorn Choe, Bumkyu Choi, Tim Faust, Habib Maraqten, Erfan Aghadavoodi Jolfaei, Marco Calì, Alberto Sinigaglia, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres, Jong-kook Kim, Gian Antonio Susto, Shubham Vyas, Dennis Mronga, Boris Belousov, Jan Peters, Frank Kirchner, Shivesh Kumar
AI summary
Problem
Benchmarking control methods on real-world robotic hardware is rare, leaving a gap in understanding how different reinforcement learning approaches handle the simulation-to-reality gap and external disturbances in dynamic, underactuated systems.
Approach
The paper evaluates four distinct RL controllers on a real double pendulum through a structured competition that tests swing-up performance and robustness against simulated and physical perturbations.
Key results
- Four diverse RL algorithms successfully executed swing-up and balance tasks on real hardware
- Controllers were rigorously evaluated on performance metrics and robustness to external perturbations
- The study highlights algorithm-specific strengths in bridging the sim-to-real gap and handling chaotic dynamics
- An open-source benchmarking framework and standardized evaluation metrics are provided for future real-robot RL research
Why it matters
Provides a practical benchmark and actionable insights for researchers developing robust, real-world reinforcement learning controllers for dynamic robotic systems.
Abstract
In robotics many different approaches ranging from classical planning over optimal control to reinforcement learning (RL) are developed and borrowed from other fields to achieve reliable control in diverse tasks. In order to get a clear understanding of their individual strengths and weaknesses and their applicability in real-world robotic scenarios it is important to benchmark and compare their performances not only in a simulation but also on real hardware. The ‘2nd AI Olympics with RealAIGym’ competition was held at the IROS 2024 conference to contribute to this cause and evaluate different controllers according to their ability to solve a dynamic control problem on an underactuated double pendulum system (Fig. 1) with chaotic dynamics. This paper describes the four different RL methods submitted by the participating teams, presents their performance in the swing-up task on a real double pendulum, measured against various criteria, and discusses their transferability from simulation to real hardware and their robustness to external disturbances.