VB-Com: Learning Vision-Blind Composite Humanoid Locomotion against Deficient Perception
humanoid robots of Unitree G and H.
AI summary
Problem
Real-world legged locomotion frequently fails when perception is noisy, delayed, or encounters untrained disturbances, yet simulating all potential perceptual deficiencies during training is impractical.
Approach
The framework trains a vision-based policy and a blind policy, using hardware-deployable return estimators to automatically switch to the blind policy when proprioceptive cues indicate perception degradation.
Key results
- Maintains >82% success rate across 0–100% graded perceptual noise
- Recovers from dynamic disturbances like rushing pedestrians and deformable gaps
- Demonstrates robust obstacle avoidance and terrain traversal on real Unitree G1 and H1 robots
- Generalizes to unseen perception failures without retraining, outperforming prior baselines
Why it matters
Enables safer, more reliable real-world deployment of humanoid robots in unstructured environments where sensors frequently fail.
Abstract
The performance of legged locomotion is highly dependent on the accuracy and completeness of state ob- servations. While perceptive locomotion enables robots to plan motions proactively and adapt to unstructured terrains, real-world perception is often degraded by sensor noise, dynamic disturbances, or training outliers. These inaccuracies can lead to locomotion failures, particularly for humanoid robots, which are especially vulnerable to misguidance from imperfect perception. However, exhaustively simulating all potential perceptual deficiencies (e.g., dynamic or deformable terrains) during training remains impractical due to inherent simulation limitations. To address this fundamental challenge, we propose VB-Com - a novel framework that dynamically combines a vision-based policy (utilizing exteroceptive sensing) with a proprioception-only blind policy through intelligent composition. When inaccurate perception begins to destabilize locomotion, VB-Com is able to identify these degradation scenarios and immediately turns to “blind" actions and recovers from the potential failure. Our approach mitigates the risks posed by deficient perception that has not be addressed by existing research on perceptive locomotion. Experimental results demonstrate that VB-Com robustly enables humanoid robots to traverse challenging terrains and obstacles–under perceptual deficiencies and dynamic disturbances. Details demonstrations can be found in our project Website: vbcom.github.io.