Freeze-Frame with StaticNeRF: Uncertainty-Guided NeRF Map Reconstruction in Dynamic Scenes
Juhui Lee, Geonmo Yang, Seungjun Ma, Younggun Cho
AI summary
Problem
Existing NeRF-based mapping methods fail to remove subtly moving objects and struggle to recover occluded static backgrounds, causing map corruption and query-map inconsistency that degrade localization reliability in dynamic environments.
Approach
StaticNeRF augments NeRF with a lightweight CNN-based uncertainty network and a three-phase curriculum learning schedule to stabilize training, compensate for unobserved static regions, and accurately identify dynamic pixels during localization.
Key results
- Robust static map reconstruction under persistent dynamic occlusion
- Improved localization accuracy in rotation and translation under dynamic conditions
- Memory-efficient architecture with multiresolution hash encoding for fast training
- Publicly available code and comprehensive evaluation on public and in-house datasets
Why it matters
Enables reliable robot navigation and visual localization in real-world environments where dynamic objects constantly change the scene.
Abstract
Recent advances in neural representations have shown great promise for enabling high-fidelity dense mapping in robotics. Given the inherently dynamic nature of real-world environments, many studies have attempted to learn static scene representations from dynamic observations. However, existing methods often fail to remove subtly moving objects and struggle to accurately recover occluded static backgrounds, which leads to critical limitations in practice. Furthermore, when static neural maps are used for localization, dynamic content in query images must be handled effectively. To overcome these challenges, we propose a static neural mapping framework that is robust to diverse dynamic environments and capable of processing dynamic content during localization. We evaluated our approach through extensive experiments on both public and in-house datasets. Our method improves both dynamic object removal and localization robustness under dynamic conditions, and constitutes a significant step toward resilient robot navigation in real-world environments.