SpikeClouds: Streaming Spike-Based Processing of LiDAR for Fast and Efficient Object Detection
Michael Neumeier, Nael Fasfous, Bing Li, Axel von Arnim
AI summary
Problem
Full-scan LiDAR processing causes high latency and large memory footprints, while existing streaming methods only support rotating mechanical scanners, not efficient scanline-based MEMS LiDARs.
Approach
The method streams LiDAR scanlines as sparse spike-based representations called SpikeClouds, processes them with a spiking neural network backbone, and feeds the accumulated features into a CNN head for object detection.
Key results
- Close to state-of-the-art detection accuracy on KITTI and JRDB22 datasets
- 10% reduction in end-to-end latency on standard GPUs
- 95% reduction in average memory footprint
- 25× lower energy consumption on neuromorphic hardware
Why it matters
Provides a scalable, low-power perception pipeline for real-time robotics and autonomous driving systems.
Abstract
LiDAR sensors are used to provide three- dimensional information about the environment in many robotics applications. The information, accumulated in 3D point clouds, is first acquired by the sensor and then processed further, which leads to high end-to-end latencies and large memory footprints. Streaming approaches tackle this problem by processing partial point cloud data during scanning of the environment. In contrast to existing work that is limited to power hungry, rotating mechanical scanners, in this paper, we present a streaming method for more efficient scanline-based LiDAR sensors. We process the sequence of scanlines in form of SpikeClouds with a Spiking Neural Network (SNN) backbone and perform 3D object detection from the accumulated information using a Convolutional Neural Network (CNN) head. Our method achieves close to state-of-the-art detection performance on datasets KITTI and JRDB22 while reducing the end-to-end latency by 10% and the average memory footprint by 95% on standard GPU hard- ware. Additionally, when ported onto neuromorphic hardware, our backbone requires 25× less energy compared to reference backbones. SpikeClouds achieves fast and efficient environmental perception for robotic applications by streaming LiDAR to enable spike-based processing.