OpenIN: Open-Vocabulary Instance-Oriented Navigation in Dynamic Domestic Environments
Yujie Tang, Meiling Wang, Yinan Deng, Zibo Zheng, Jingchuan Deng, Sibo Zuo, Yufeng Yue
AI summary
Problem
Current object navigation methods lack dynamic scene representation updates, making it difficult for robots to accurately track and navigate to frequently moved instances in domestic environments.
Approach
OpenIN constructs a dynamic Carrier-Relationship Scene Graph (CRSG) to track objects and their carriers, continuously updating it during navigation, and employs an MDP-based strategy guided by visual-language features and LLM commonsense to direct instance-level searches.
Key results
- Supports multi-type navigation instructions including demands, semantics, and instance descriptions
- Dynamically updates carrier-carried relationships and spatial edges during robot movement
- Leverages visual-language similarity and LLM commonsense for accurate target prioritization
- Demonstrates improved navigation efficiency and accuracy in both simulation and real-robot tests
Why it matters
Advances practical domestic robotics by enabling reliable, long-horizon search for everyday items in constantly changing home environments.
Abstract
In daily domestic settings, frequently used objects like cups often have unfixed positions and multiple instances within the same category, and their carriers also frequently change. As a result, it becomes challenging for a robot to effi- ciently navigate to a specific instance. To tackle this challenge, the robot must capture and update scene changes and plans contin- uously. However, current object-navigation approaches primarily focus on the semantic level and lack the ability to dynamically update the scene representation. In contrast, this paper captures the relationships between frequently used objects and their static carriers. It constructs an open-vocabulary Carrier-Relationship Scene Graph (CRSG) and updates the carrying status during robot navigation to reflect the dynamic changes of the scene. Based on CRSG, we further propose an instance navigation strategy that models the navigation process as a Markov Decision Process. At each step, decisions are informed by the Large Language Model’s commonsense knowledge and visual-language feature similarity. We designed a series of long-horizon navigation tasks for frequently used everyday items in the Habitat simula- tor. The results demonstrate that by updating the CRSG, the robot can navigate efficiently to moved targets. Additionally, we conducted extensive experiments on a real robot, demonstrating the effectiveness of our method and exploring its limitations. The project page can be found here: https://OpenIN-nav.github.io. Manuscript received: January 6, 2025; Revised: May 31, 2025; Accepted: June 30, 2025. This paper was recommended for publication by Editor Ashis Banerjee upon evaluation of the Associate Editor and Reviewers’ comments. This work was supported by the National Natural Science Foundation of China under Grant No. NSFC 62473050, 92370203, 62233002. (Corresponding Author: Yufeng Yue, yueyufeng@bit.edu.cn) 1Yujie Tang, Meiling Wang, Yinan Deng, Jingchuan Deng, Sibo Zuo, Yufeng Yue are with School of Automation, Beijing Institute of Technology, Beijing, 100081, China. 2Zibo Zheng is with School of Mechanical Engineering, University of Nottingham Ningbo China, Ningbo, 315100, China. Digital Object Identifier (DOI): see top of this page.