DualMap: Online Open-Vocabulary Semantic Mapping for Natural Language Navigation in Dynamic Changing Scenes
Jiajun Jiang, Yiming Zhu, Zirui Wu, Jie Song
AI summary
Problem
Existing semantic mapping systems struggle to simultaneously support open-vocabulary queries, efficient online updates, and navigation in dynamically changing environments, limiting their real-world applicability.
Approach
The system builds a detailed 3D concrete map using a hybrid segmentation frontend, then abstracts it into a global map of static anchors for high-level planning, allowing online updates and re-navigation when objects move.
Key results
- State-of-the-art performance in 3D open-vocabulary segmentation and online mapping
- Elimination of costly 3D inter-object merging via lightweight intra-object status checks
- Real-time language-guided navigation that adapts to dynamic object relocations
- Validated across simulation benchmarks and real-world indoor/outdoor experiments
Why it matters
Enables practical, lifelong robotic navigation in human-centered spaces where objects frequently move and queries are open-ended.
Abstract
We introduce DualMap, an online open-vocabulary mapping system that enables robots to understand and navigate dynamically changing environments through natural language queries. Designed for efficient semantic mapping and adaptability to changing environments, DualMap meets the essential requirements for real-world robot navigation applications. Our proposed hybrid segmentation frontend and object-level status check eliminate the costly 3D object merging required by prior methods, enabling efficient online scene mapping. The dual-map representation combines a global abstract map for high-level candidate selection with a local concrete map for precise goal-reaching, effectively managing and updating dynamic changes in the environment. Through extensive experiments in both simulation and real-world scenarios, we demonstrate state- of-the-art performance in 3D open-vocabulary segmentation, efficient scene mapping, and online language-guided navigation. Project page: https://eku127.github.io/DualMap/