Receding Horizon Reinforcement Learning with Autoregressive Model for Motion Control of Autonomous Vehicles
Xin Yin, Haotian CAO, Xinglong Zhang, Qingwen Ma, Xin Xu, Haibin Xie
AI summary
Problem
Traditional vehicle control methods struggle to balance computational efficiency with accurate real-time prediction of complex dynamics and environmental uncertainties, limiting lateral trajectory tracking performance on unstructured roads.
Approach
The authors propose a model-based reinforcement learning framework that uses an online-updated autoregressive model to predict and compensate for unmodeled vehicle dynamics and disturbances in real-time, paired with an actor-critic neural network for control optimization.
Key results
- Developed MBRL-AR framework with receding horizon mechanism for real-time state-feedback control
- Created dynamic residual generation mechanism to compensate for nonlinear tire behavior and external disturbances
- Achieved superior tracking accuracy and adaptability in CarSim simulations over RHACL, DRHACL, MPC, and LQR
- Validated on a real-world HongQi electric vehicle, demonstrating significant error reduction and resilience over classical MPC
Why it matters
Provides a computationally efficient and highly adaptive control solution for autonomous vehicles, accelerating the deployment of robust advanced driver-assistance and fully autonomous systems in complex real-world environments.
Abstract
This paper presents a model-based reinforcement learning (MBRL) approach with a receding horizon mechanism to optimize the lateral trajectory-tracking performance of autonomous vehicles (AVs). Accurate modeling of complex vehicle dynamics and adaptation to dynamic environments with limited data pose significant challenges for MBRL in AV control. To address these challenges, we propose sample-efficient algorithms that leverage autoregressive modeling to adapt from limited data while managing complex vehicle dynamics. Unlike traditional methods reliant on fixed models, our approach uses the temporal reasoning of autoregressive (AR) models to compensate for the residual dynamics, which effectively ap- proximates the local effects of nonlinearities and disturbances. Integrated with real-time sensor data, the residual generation model is continuously refined via incremental learning in a closed-loop framework, enhancing adaptability. This architec- ture, combining physical modeling with data-driven residuals, maintains interpretability and improves responsiveness in com- plex scenarios. CarSim simulations demonstrate superior per- formance over other state-of-the-art learning-based predictive controllers and classical methods for AV lateral control. Real- world validation on a HongQi electric vehicle (HQEV) confirms the algorithm’s effectiveness, showing significant improvements over classical model predictive control (MPC). This approach holds substantial potential for advanced driver-assistance sys- tems (ADAS) and fully autonomous driving, enabling precise control under diverse conditions.