Behavior Foundation Model for Humanoid Robots
Weishuai Zeng, Shunlin Lu, Kangning Yin, Xiaojie Niu, Minyue Dai, Jingbo Wang, Jiangmiao Pang
AI summary
Problem
Existing whole-body control frameworks are rigidly task-specific, requiring labor-intensive reward engineering and failing to generalize across different control modes or tasks, which limits real-world deployment.
Approach
The authors introduce BFM, a generative model pretrained on large-scale behavioral datasets using a masked online distillation framework with a Conditional Variational Autoencoder (CVAE) to capture reusable behavioral knowledge and support flexible control mode steering.
Key results
- Zero-shot execution of diverse whole-body tasks across arbitrary control modes
- Rapid acquisition of novel behaviors via residual learning without full retraining
- Robust generalization validated in both simulation and real-world humanoid platforms
- Unified behavioral formulation that decouples control modes from task objectives
Why it matters
Offers a scalable, general-purpose foundation for humanoid control that reduces task-specific tuning and accelerates skill acquisition for complex robotic applications.
Abstract
Whole-body control (WBC) of humanoid robots has witnessed remarkable progress in skill versatility, enabling a wide range of applications such as locomotion, teleoperation, and motion tracking. Despite these achievements, existing WBC frameworks remain largely task-specific, relying heavily on labor-intensive reward engineering and demonstrating limited generalization across tasks and skills. These limitations hinder their response to arbitrary control modes and restrict their deployment in complex, real-world scenarios. To address these challenges, we revisit existing WBC systems and identify a shared objective across diverse tasks: the generation of ap- propriate behaviors that guide the robot toward desired goal states. Building on this insight, we propose the Behavior Foun- dation Model (BFM), a generative model pretrained on large- scale behavioral datasets to capture broad, reusable behavioral knowledge for humanoid robots. BFM integrates a masked online distillation framework with a Conditional Variational Autoencoder (CVAE) to model behavioral distributions, thereby enabling flexible operation across diverse control modes and efficient acquisition of novel behaviors without retraining from scratch. Extensive experiments in both simulation and on a physical humanoid platform demonstrate that BFM generalizes robustly across diverse WBC tasks while rapidly adapting to new behaviors. These results establish BFM as a promising step toward a foundation model for general-purpose humanoid control. Videos and supplementary materials are available at bfm4humanoid.github.io.