HMC: Learning Heterogeneous Meta-Control for Contact-Rich Loco-Manipulation
whiteboard; pulling open a door; opening a microwave; tightening a nut and lifting a trash can.)
AI summary
Problem
Purely positional controllers fail during contact-rich interactions due to neglected dynamics, while existing compliance methods lack generalizability and struggle with data imbalance and abrupt controller switching.
Approach
HMC introduces a low-level interface that continuously blends torque outputs from multiple control modalities, paired with a high-level heterogeneous policy that uses soft mixture-of-experts routing and two-stage training to adaptively weight controllers.
Key results
- Unified low-level HMC-Controller enabling continuous torque-space blending of position, impedance, and hybrid force controllers
- HMC-Policy architecture with soft Mixture-of-Experts routing and pretrain-finetune strategy to mitigate data imbalance
- Over 50% relative improvement in task success rates on real-world contact-rich loco-manipulation tasks compared to baselines
- Demonstrated stability and adaptability across table wiping, bimanual bottle lifting, and magnetic drawer opening
Why it matters
Enables safe, robust, and generalizable whole-body robot interaction in unstructured environments where precise force and motion regulation are critical.
Abstract
Learning from real-world robot demonstrations holds promise for interacting with complex real-world envi- ronments. However, the complexity and variability of inter- action dynamics often cause purely positional controllers to struggle with contacts or varying payloads. To address this, we propose a Heterogeneous Meta-Control (HMC) framework for Loco-Manipulation that adaptively stitches multiple control modalities: position, impedance, and hybrid force-position. We first introduce an interface, HMC-Controller, for blending actions from different control profiles continuously in the torque space. HMC-Controller facilitates both teleoperation and policy deployment. Then, to learn a robust force-aware policy, we propose HMC-Policy to unify different controllers into a heterogeneous architecture. We adopt a mixture-of- experts style routing to learn from large-scale position-only data and fine-grained force-aware demonstrations. Experiments on a real humanoid robot show over 50% relative improvement vs. baselines on challenging tasks such as compliant table wiping and drawer opening, demonstrating the efficacy of HMC.