Unveiling the Impact of Data and Model Scaling on High-Level Control for Humanoid Robots
Yuxi Wei, Ziseoi Wong, Kangning Yin, Yue Hu, Jingbo Wang, Siheng Chen
AI summary
Problem
Existing humanoid robot learning struggles with scarce, low-quality data and lacks effective scalable learning strategies, while relying on direct human motion priors causes modality misalignment and execution failures.
Approach
The authors introduce Humanoid-Union, a 260-hour dataset of robot motions automatically derived and filtered from human videos, and propose SCHUR, a scalable text-to-motion generation model using Finite Scalar Quantization and a LLaMA-based architecture.
Key results
- 260+ hour dataset of human-derived, tracker-filtered robot motions
- 37% MPJPE and 25% FID improvements over prior methods
- Continuous quality gains from scaling codebook size and model parameters
- Successful real-world deployment of text-conditioned whole-body control
Why it matters
Provides a scalable, data-driven foundation for high-level text control of humanoid robots, bridging the gap between human motion priors and executable robotic control.
Abstract
Data scaling has long remained a critical bottle- neck in robot learning. For humanoid robots, human videos and motion data are abundant and widely available, offering a free and large-scale data source. Besides, the semantics related to the motions enable modality alignment and high-level robot control learning. However, how to effectively mine raw video, extract robot-learnable representations, and leverage them for scalable learning remains an open problem. To address this, we introduce Humanoid-Union, a large-scale dataset generated through an autonomous pipeline, comprising over 260 hours of diverse, high-quality humanoid robot motion data with semantic annotations derived from human motion videos. The dataset can be further expanded via the same pipeline. Building on this data resource, we propose SCHUR, a scalable learn- ing framework designed to explore the impact of large-scale data on high-level control in humanoid robots. Experimental results demonstrate that SCHUR achieves high robot motion generation quality and strong text-motion alignment under data and model scaling, with 37% reconstruction improvement under MPJPE and 25% alignment improvement under FID comparing with previous methods. Its effectiveness is further validated through deployment in real-world humanoid robot.