SLIM-VDB: A Real-Time 3D Probabilistic Semantic Mapping Framework
Anja Sheppard, Parker Ewen, Joseph Wilson, Advaith Venkatramanan Sethuraman, Benard Adewole, Anran Li, Yuzhen Chen, Ram Vasudevan, Katherine Skinner
AI summary
Problem
Existing semantic mapping systems struggle with real-time performance and high memory demands, and they typically support either closed-set or open-set semantics but lack a unified probabilistic framework to handle both.
Approach
The method leverages the OpenVDB sparse volumetric data structure and introduces a unified Bayesian update mechanism that recursively fuses semantic predictions from fixed-category and open-language models into a single probabilistic map.
Key results
- First OpenVDB-based semantic mapping framework for robotics
- Unified Bayesian fusion supporting closed- and open-set semantics
- Significant reduction in memory usage and integration time versus state-of-the-art
- Open-source C++ library with Python interface for robotics integration
Why it matters
Enables resource-constrained mobile robots to build accurate, real-time semantic maps using either traditional or modern vision-language models, advancing scalable scene understanding.
Abstract
This letter introduces SLIM-VDB, a new lightweight semantic mapping system with probabilistic semantic fusion for closed-set or open-set dictionaries. Advances in data structures from the computer graphics community, such as OpenVDB, have demonstrated significantly improved computational and memory efficiency in volumetric scene representation. Although OpenVDB has been used for geometric mapping in robotics applications, se- mantic mapping for scene understanding with OpenVDB remains unexplored. In addition, existing semantic mapping systems lack support for integrating both fixed-category and open-language la- bel predictions within a single framework. In this letter, we propose a novel 3D semantic mapping system that leverages the OpenVDB data structure and integrates a unified Bayesian update framework for both closed- and open-set semantic fusion. Our proposed frame- work, SLIM-VDB, achieves significant reduction in both memory and integration times compared to current state-of-the-art seman- tic mapping approaches, while maintaining comparable mapping accuracy.