VGGT-Long: Chunk It, Loop It, Align It -- Pushing VGGT's Limits on Kilometer-Scale Long RGB Sequences
Kai Deng, Zexin Ti, Xu Jiawei, Jian Yang, Jin Xie
AI summary
Problem
Transformer-based 3D foundation models face prohibitive GPU memory limits and accumulated drift when processing long, uncalibrated RGB sequences, preventing their use in large-scale outdoor reconstruction.
Approach
The system divides long sequences into overlapping chunks processed independently by VGGT, aligns adjacent chunks using confidence-weighted Sim(3) transformations, and corrects global drift via a lightweight loop closure and Levenberg-Marquardt optimization.
Key results
- Successful kilometer-scale monocular 3D reconstruction from uncalibrated RGB streams
- Trajectory and geometry accuracy comparable to traditional calibrated methods
- Elimination of camera calibration, depth supervision, and model retraining requirements
- Efficient GPU/CPU memory management enabling long-sequence processing on standard hardware
Why it matters
Provides a scalable, minimalist pathway to deploy powerful foundation models for real-world autonomous driving and large-scale 3D perception without complex SLAM backends.
Abstract
Foundation models for 3D vision have recently demonstrated remarkable capabilities in 3D perception. How- ever, extending these models to large-scale RGB stream 3D reconstruction remains challenging due to memory limitations. In this work, we propose VGGT-Long, a simple yet effective system that pushes the limits of monocular 3D reconstruction to kilometer-scale, unbounded outdoor environments. Our ap- proach addresses the scalability bottlenecks of existing models through a chunk-based processing strategy combined with overlapping alignment and lightweight loop closure optimiza- tion. Without requiring camera calibration, depth supervision or model retraining, VGGT-Long achieves trajectory and re- construction performance comparable to traditional methods. We evaluate our method on KITTI, Waymo, and Virtual KITTI datasets. VGGT-Long not only runs successfully on long RGB sequences where foundation models typically fail, but also produces accurate and consistent geometry across various conditions. Our results highlight the potential of leveraging foundation models for scalable monocular 3D scene in real- world settings, especially for autonomous driving scenarios. Code is available at https://github.com/DengKaiCQ/ VGGT-Long.