UrbanVLA: A Vision-Language-Action Model for Urban Micromobility
Anqi Li, Zhiyong Wang, Jiazhao Zhang, Minghan Li, Yunpeng Qi, Zhibo Chen, Zhizheng Zhang, He Wang
AI summary
Problem
Existing urban navigation methods struggle with scalability and robustness in dynamic, unstructured city environments due to reliance on costly maps or coarse, misaligned route waypoints from navigation apps.
Approach
The authors propose UrbanVLA, a route-conditioned Vision-Language-Action model that aligns noisy high-level route instructions with real-time visual inputs, trained via supervised fine-tuning on simulation and web data followed by offline reinforcement fine-tuning to enhance safety and real-world adaptability.
Key results
- Surpasses baselines by >55% in MetaUrban SocialNav benchmarks
- Achieves reliable zero-shot real-world navigation across diverse urban settings
- Enables long-horizon route following spanning over 500 meters
- Introduces a sim-to-real pipeline with heuristic trajectory lifting for scalable training
Why it matters
It provides a scalable, map-free navigation framework that bridges consumer navigation apps with safe, real-world deployment for delivery robots and other urban micromobility platforms.
Abstract
Urban micromobility applications, such as de- livery robots, demand reliable navigation across large-scale urban environments while following long-horizon route in- structions. This task is particularly challenging due to the dynamic and unstructured nature of real-world city areas, yet most existing navigation methods remain tailored to short- scale and controllable scenarios. Effective urban micromobility requires two complementary levels of navigation skills: low-level capabilities such as point-goal reaching and obstacle avoidance, and high-level capabilities, such as route–visual alignment. To this end, we propose UrbanVLA, a route-conditioned Vision- Language-Action (VLA) framework designed for scalable urban navigation. Our method explicitly aligns noisy route waypoints with visual observations during execution, and subsequently plans trajectories to drive the robot. To enable UrbanVLA to master both levels of navigation, we employ a two-stage training pipeline. The process begins with Supervised Fine- Tuning (SFT) using simulated environments and trajectories parsed from web videos. This is followed by Reinforcement Fine-Tuning (RFT) on a mixture of simulation and real-world data, which enhances the model’s safety and adaptability in real-world settings. Experiments demonstrate that UrbanVLA surpasses strong baselines by more than 55% in the SocialNav task in MetaUrban. Furthermore, UrbanVLA achieves reliable real-world navigation, showcasing both scalability to large- scale urban environments and robustness against real-world uncertainties.