FAST-LIEO: Fast and Real-Time LiDAR-Inertial-Event-Visual Odometry
Zirui Wang, Yangtao Ge, Kewei Dong, I-Ming Chen, Jing Wu
AI summary
Problem
Traditional cameras suffer from motion blur and exposure issues in high dynamic range scenes, while few multi-sensor fusion methods effectively integrate event cameras with LiDAR and inertial sensors for reliable state estimation.
Approach
The framework tightly couples LiDAR, IMU, event, and optional RGB measurements through three subsystems linked by an error-state Kalman filter, using a novel event-inertial odometry module that aligns time-surface maps to a LiDAR-derived semi-dense map without feature extraction.
Key results
- First system to tightly fuse LiDAR, IMU, event, and RGB camera measurements
- Novel EIO subsystem enables real-time alignment without feature extraction
- High robustness and accuracy on public benchmarks and self-collected dataset
- Flexible LIEO and LIEVO configurations with optional visual fusion
Why it matters
Provides a reliable navigation solution for autonomous robots and vehicles operating in challenging environments where standard cameras fail.
Abstract
Unlike a standard camera that relies on exposure to obtain output frame by frame, an event camera only outputs an event when the change of brightness intensity in a pixel exceeds a threshold, and the outputs of different pixels are independent to each other. Benefited from its bio-inspired design, event camera has the advantages of low latency and high dynamic range. The researches on multi-sensor fusion with event camera are few so far. In this paper, we propose FAST-LIEO, a framework for fast and real-time LiDAR-inertial-event odometry. The framework tightly fuses LiDAR and event camera measurements without any feature extraction or matching. Besides, our system supports both LIEO and LIEVO (extended with RGB camera fusion). We design a novel EIO subsystem for LiDAR-event fusion. The EIO subsystem main- tains a semi-dense event map and estimates the state by aligning the event representation to map. The semi-dense event map is built from LiDAR points by utilizing the edge information and temporal information provided by event representations. Besides testing our method on public benchmark dataset, we also collected real-world data by utilizing our sensor suite and conducted experiments on our self-captured dataset. The experiment results show the high robustness and accuracy of our method in challenging conditions with high real-time ability. To the best of our knowledge, our FAST-LIEO is the first system that can tightly fuse LiDAR, IMU, event camera and standard camera measurements in simultane- ously localization and mapping.