Data-Efficient Constrained Robot Learning with Probabilistic Lagrangian Control
Shiming He, Yuzhe Ding
AI summary
Problem
Lagrangian methods in constrained robot learning often exhibit oscillatory primal-dual updates that cause safety violations and slow convergence. Existing Bayesian optimization techniques fail to scale to high-dimensional policy spaces while maintaining strict safety guarantees.
Approach
The method combines gradient-information Bayesian optimization with a Jacobian Gaussian process model, using the posterior probability of constraint satisfaction to actively minimize the Lagrange multiplier while ensuring safety.
Key results
- Reduces oscillatory dynamics in multiplier updates
- Achieves lower regret in high-dimensional synthetic domains
- Finds feasible policies in simulated and real robot tasks
- Scales to 64-dimensional policy spaces without manual tuning
Why it matters
Enables safe, sample-efficient deployment of learning-based controllers on physical robots where trial-and-error is costly and hazardous.
Abstract
We propose a novel framework for data-efficient black-box robot learning under constraints. Our approach in- tegrates probabilistic inference with Lagrangian optimization. With the guide of a learned Gaussian process model, the La- grange multiplier is controlled by the probability of whether the constraints would be satisfied. This reduces the typical oscillations seen in primal-dual updates and therefore improves both data efficiency and safety during learning. Both synthetic results and robot experiments demonstrate that our method is a scalable and effective solution for constrained robot learning problems.