UnIRe: Unsupervised Instance Decomposition for Dynamic Urban Scene Reconstruction
Yunxuan Mao, Rong Xiong, Yue Wang, Yiyi Liao
AI summary
Problem
Existing dynamic scene reconstruction methods either rely on expensive manual bounding box annotations or fail to decompose dynamic objects into individual instances, hindering scalable, object-level scene editing.
Approach
UnIRe decomposes scenes into static and dynamic components using unsupervised 4D superpoint clustering on LiDAR sequences, then initializes dynamic 3D Gaussian Splatting with a canonical space and per-point deformation, stabilized by 2D and 3D smoothness regularization.
Key results
- State-of-the-art novel view synthesis and reconstruction on Waymo and KITTI without annotations
- Accurate instance-level scene editing (e.g., vehicle removal, pedestrian addition) without bounding boxes
- 4D superpoint clustering resolves over/under-decomposition and inconsistent object tracking across frames
- Reduced overfitting and improved motion consistency via novel 2D and 3D smoothness regularization
Why it matters
Provides a scalable, annotation-free pipeline for dynamic urban scene understanding, directly benefiting autonomous driving simulation, urban planning, and interactive environment editing.
Abstract
Reconstructing and decomposing dynamic urban scenes is crucial for autonomous driving, urban planning, and scene editing. However, existing methods fail to perform instance-aware decomposition without manual annotations, which is crucial for instance-level scene editing. We propose UnIRe, a 3D Gaussian Splatting (3DGS) based approach that decomposes a scene into a static background and individual dynamic instances using only RGB images and LiDAR point clouds. At its core, we introduce 4D superpoints, a novel representation that clusters multi-frame LiDAR points in 4D space, enabling unsupervised instance separation based on spatiotemporal correlations. These 4D superpoints serve as the foundation for our decomposed 4D initialization, i.e., providing spatial and temporal initialization to train a dynamic 3DGS for arbitrary dynamic classes without requiring bounding boxes or object templates. Furthermore, we introduce a smoothness regularization strategy in both 2D and 3D space, further improving the temporal stability. Experiments on benchmark datasets show that our method outperforms existing methods in decomposed dynamic scene reconstruction while enabling accurate and flexible instance-level editing, making it a practical solution for real-world applications.