Neural Internal Model Control: Learning a Robust Control Policy Via Predictive Error Feedback
Feng Gao, Chao Yu, Yu Wang, Yi Wu
AI summary
Problem
Classical model-based controllers struggle with nonlinear dynamics and unstructured disturbances, while reinforcement learning policies often lack robustness in unseen scenarios. Effectively bridging these approaches without complex tuning or precise disturbance estimation remains a challenge.
Approach
NeuralIMC uses a simplified Newton-Euler rigid-body model to predict the next robot state and feeds the prediction error as feedback into a model-free RL policy. This closed-loop structure compensates for modeling inaccuracies and external disturbances without requiring handcrafted estimators.
Key results
- Surpasses state-of-the-art tracking accuracy for quadrotors under randomized wind disturbances
- Achieves superior adaptive locomotion on complex terrains for quadrupedal robots
- Demonstrates robust sim-to-real transfer on a physical quadrotor with suspended payloads
- Provides a plug-and-play feedback loop compatible with existing RL algorithms
Why it matters
Provides a practical, tuning-light framework for deploying robust robotic controllers in dynamic real-world applications, benefiting both aerial and legged robotics researchers.
Abstract
Accurate motion control in the face of distur- bances within complex environments remains a major challenge in robotics. Classical model-based approaches often struggle with nonlinearities and unstructured disturbances, while rein- forcement learning (RL)-based methods can be fragile when encountering unseen scenarios. In this paper, we propose a novel framework, Neural Internal Model Control (NeuralIMC), which integrates model-based control with RL-based control to enhance robustness. Our framework streamlines the pre- dictive model by applying Newton-Euler equations for rigid- body dynamics, eliminating the need to capture complex high- dimensional nonlinearities. This internal model combines model- free RL algorithms with predictive error feedback. Such a design enables a closed-loop control structure to enhance the robustness and generalizability of the control system. We demonstrate the effectiveness of our framework on both quadrotors and quadrupedal robots, achieving superior performance compared to state-of-the-art methods. Furthermore, real-world deployment on a quadrotor with rope-suspended payloads highlights the frameworkâs robustness in sim-to-real transfer. Our code is released at https://github.com/thu-uav/NeuralIMC.