Long-Horizon Planning with Large Language Models for Indoor Assistive Navigation of the Visually Impaired
Yiyang Sun, Chengran Lin, Ji Xia, Zhengcai Cao
AI summary
Problem
Visually impaired individuals struggle with complex indoor navigation due to the inability of traditional planners to interpret natural language instructions and the susceptibility of LLMs to geometric hallucinations. Existing assistive tools often restrict user autonomy or lack hands-free operation.
Approach
The system employs an LLM agent to interpret user instructions and generate high-level semantic routes grounded in a topological map, while delegating precise node-level path computation to a classical graph search algorithm. A wearable device then provides hands-free guidance through voice cues and real-time vibrotactile feedback.
Key results
- Novel LLM-based planning agent for grounded long-horizon route generation
- Decoupled architecture shielding LLMs from geometric reasoning errors via external graph search
- Wearable assistive device enabling hands-free navigation with voice and vibrotactile feedback
- Successful real-world validation in hospital environments demonstrating reliable autonomous navigation
Why it matters
Enables independent, hands-free indoor mobility for the visually impaired by bridging natural language intent with reliable navigation planning.
Abstract
For visually impaired individuals, assistive naviga- tion systems play a crucial role in enabling independent mobil- ity. However, long-horizon planning based on natural language (NL) instructions in complex indoor environments remains a significant challenge. Recent studies show the strong potential of Large Language Models (LLMs) in NL understanding and task-level planning. Yet, the inherent limitations of LLMs in mathematical reasoning and their susceptibility to hallucination hinder their reliability in low-level path planning. In this paper, we introduce an LLM-based indoor assistive navigation system that interprets NL instructions from visually impaired users for autonomous navigation. At its core is a novel planning agent that grounds instructions to the environment’s topological map and generates optimal route plans. To avoid hallucination in geometric reasoning, the LLM handles only high-level semantic planning, while precise node-level paths are delegated to a classical graph search algorithm. We further implement a wearable assistive device that provides voice and vibrotactile feedback to deliver hands-free navigation. Offline evaluations and real-world experiments demonstrate that our system can reliably plan grounded routes and enable visually impaired users to autonomously complete long-horizon navigation tasks. Anonymous project page is available at https://lhp-ian.github.io.