← Back ICRA 2026

LaViRA: Language-Vision-Robot Actions Translation for Zero-Shot Vision Language Navigation in Continuous Environments

Hongyu Ding, Ziming Xu, YUK TUNG SAMUEL FANG, You Wu, Zixuan Chen, Jieqi Shi, Jing Huo, Yifan Zhang, Yang Gao

PDF

AI summary

Key figure (auto-extracted from paper)

LaViRA achieves state-of-the-art zero-shot navigation in unseen environments by hierarchically decomposing decisions into language planning, vision grounding, and robot control.

Zero-shot navigation Vision-language navigation Multimodal LLMs Hierarchical planning Continuous environments Robot control

Problem

Current zero-shot vision-and-language navigation methods face a trade-off: waypoint predictors limit generalization to unseen scenes, while value-mapping approaches underutilize large model reasoning during online navigation.

Approach

LaViRA decomposes navigation into a three-stage hierarchy: a large MLLM handles high-level language planning, a smaller MLLM grounds the plan to a visual target, and a rule-based controller executes low-level robot movement.

Key results

Sets new state-of-the-art for zero-shot VLN-CE on the Habitat benchmark
Achieves 38.3% success rate and 28.3 SPL on unseen environments
Validates multi-scale MLLM pairing for optimal performance and cost
Maintains low inference cost and high robustness across runs

Why it matters

Enables reliable, training-free navigation for real-world robots by leveraging multi-scale MLLM reasoning without environment-specific tuning.

Abstract

Zero-shot Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires an agent to navigate unseen environments based on natural language in- structions without any prior training. Current methods face a critical trade-off: either rely on environment-specific waypoint predictors that limit scene generalization, or underutilize the reasoning capabilities of large models during navigation. We introduce LaViRA, a simple yet effective zero-shot framework that addresses this dilemma by decomposing action into a coarse-to-fine hierarchy: Language Action for high-level plan- ning, Vision Action for middle-level perceptual grounding, and Robot Action for low-level control. This modular decomposition allows us to leverage the distinct strengths of different scales of Multimodal Large Language Models (MLLMs) at each stage, creating a system that is powerful in its reasoning, grounding and practical control. LaViRA significantly out- performs existing state-of-the-art methods on the VLN-CE benchmark, demonstrating superior generalization capabilities in unseen environments, while maintaining transparency and efficiency for real-world deployment. Project page: https: //robo-lavira.github.io/lavira-zs-vln/

Index terms

Visual Learning Vision-Based Navigation RGB-D Perception