Look Forward to Walk Backward: Efficient Terrain Memory for Backward Locomotion with Forward Vision
Shixin Luo, Songbo Li, Yuan Hao, Yaqi Wang, Jun Zheng, Jun Wu, Qiuguo Zhu
AI summary
Problem
Legged robots with forward-facing cameras lack terrain preview when walking backward, causing collisions or poor performance with purely proprioceptive controllers. Existing memory or mapping approaches add latency, accumulate drift, or require excessive GPU memory, making them unsuitable for real-time onboard control.
Approach
The framework trains a controller to encode forward depth and proprioception into a compact, associative terrain memory using a delta-rule recurrent network. This memory is retrieved during backward motion to anticipate obstacles, enabling agile navigation without rearward cameras or heavy mapping modules.
Key results
- Achieves collision-free backward locomotion on complex step and gap terrains using only forward vision
- Enables constant-time, constant-memory per-step inference suitable for low-cost onboard processors
- Outperforms LSTM, Transformer-XL, and linear attention baselines in backward elevation estimation accuracy and success rate
- Demonstrates zero-shot real-world deployment on a quadruped robot
Why it matters
Enables resource-constrained legged robots to perform agile backward maneuvers safely, expanding their operational versatility in unstructured environments without requiring expensive rearward sensors or heavy compute.
Abstract
Legged robots with egocentric forward-facing depth cameras can couple exteroception and proprioception to achieve robust forward agility on complex terrain. When these robots walk backward, the forward-only field of view provides no preview. Purely proprioceptive controllers can remain stable on moderate ground when moving backward but cannot fully exploit the robot’s capabilities on complex terrain and must collide with obstacles. We present Look Forward to Walk Backward (LF2WB), an efficient terrain-memory locomotion framework that uses forward egocentric depth and proprioception to write a compact associative memory during forward motion and to retrieve it for collision-free backward locomotion without rearward vision. The memory backbone employs a delta-rule selective update that softly removes then writes the memory state along the active subspace. Training uses hardware-efficient parallel computation, and deployment runs recurrent, constant-time per-step inference with a constant-size state, making the approach suitable for onboard processors on low-cost robots. Experiments in both simulations and real- world scenarios demonstrate the effectiveness of our method, improving backward agility across complex terrains under limited sensing.