Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking
João Araújo, Yanjie Ze, Pei Xu, Jiajun Wu, Karen Liu
AI summary
Problem
Humanoid motion tracking policies face an embodiment gap that forces them to imitate physically infeasible motions when standard retargeting introduces artifacts like foot sliding and penetration.
Approach
The authors introduce General Motion Retargeting (GMR), which applies non-uniform local scaling and two-stage optimization to eliminate artifacts, and rigorously evaluate its impact on RL policy success using the BeyondMimic framework.
Key results
- Retargeting artifacts significantly reduce policy robustness for dynamic or long sequences
- GMR outperforms open-source retargeters in tracking accuracy and motion faithfulness
- GMR achieves success rates and perceptual fidelity close to a closed-source baseline
- Foot penetration, self-intersection, and velocity spikes are critical artifacts to avoid
Why it matters
This work establishes retargeting quality as a fundamental bottleneck in humanoid teleoperation and imitation learning, guiding researchers to prioritize physically feasible reference data.
Abstract
Humanoid motion tracking policies are central to building teleoperation pipelines and hierarchical controllers, yet they face a fundamental challenge: the embodiment gap between humans and humanoid robots. Current approaches address this gap by retargeting human motion data to hu- manoid embodiments and then training reinforcement learning (RL) policies to imitate these reference trajectories. However, artifacts introduced during retargeting, such as foot sliding, self-penetration, and physically infeasible motion are often left in the reference trajectories for the RL policy to correct. While prior work has demonstrated motion tracking abilities, they often require extensive reward engineering and domain randomization to succeed. In this paper, we systematically evaluate how retargeting quality affects policy performance when excessive reward tuning is suppressed. To address issues that we identify with existing retargeting methods, we propose a new retargeting method, General Motion Retargeting (GMR). We evaluate GMR alongside two open-source retargeters, PHC and ProtoMotions2, as well as with a high-quality closed-source dataset from Unitree. Using BeyondMimic for policy training, we isolate retargeting effects without reward tuning. Our experiments on a diverse subset of the LAFAN1 dataset reveal that while most motions can be tracked, artifacts in retargeted data significantly reduce policy robustness, particularly for dynamic or long sequences. GMR consistently outperforms existing open-source methods in both tracking performance and faithfulness to the source motion, achieving perceptual fidelity and policy success rates close to the closed-source base- line. Website: jaraujo98.github.io/retargeting matters. Code: github.com/YanjieZe/GMR.