Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model
Minho Lee, Hyeonseok Kim, Jin Tak Kim, Sangshin Park, Jeong Hyun Lee, Jungsan Cho, Jemin Hwangbo
AI summary
Problem
Sim-to-real transfer for large-scale hydraulic robots is hindered by complex fluid dynamics, slow control responses, and the high risk of collecting real-world data, making reinforcement learning impractical.
Approach
The authors develop a simplified analytical actuator model that captures hydraulic dynamics and predicts joint torques in under one microsecond, enabling fast and accurate reinforcement learning simulations.
Key results
- Analytical actuator model predicts torques in <1µs with high sim-to-real fidelity
- RL locomotion policy successfully deployed on a 300+ kg hydraulic quadruped
- Stable command-tracking locomotion achieved at 1 m/s with dynamic adaptability
- Outperforms neural network-based models in data-limited scenarios
Why it matters
Overcomes critical sim-to-real barriers for heavy hydraulic legged robots, advancing their deployment in industrial and defense applications.
Abstract
The simulation-to-reality (sim-to-real) transfer of large-scale hydraulic robots presents a significant challenge in robotics because of the inherent slow control response and complex fluid dynamics. The complex dynamics result from the multiple interconnected cylinder structure and the difference in fluid rates of the cylinders. These characteristics complicate detailed simulation for all joints, making it unsuitable for rein- forcement learning (RL) applications. In this work, we propose an analytical actuator model driven by hydraulic dynamics to represent the complicated actuators. The model predicts joint torques for all 12 actuators in under 1 microsecond, allowing rapid processing in RL environments. We compare our model with neural network-based actuator models and demonstrate the advantages of our model in data-limited scenarios. The locomotion policy trained in RL with our model is deployed on a hydraulic quadruped robot, BeTheX-Q, which is over 300 kg. This work is the first demonstration of a successful transfer of stable and robust command-tracking locomotion with RL on a heavy hydraulic quadruped robot, demonstrating advanced sim- to-real transferability.