A Photorealistic Dataset and Vision-Based Algorithm for Anomaly Detection During Proximity Operations in Lunar Orbit
Selina Leveugle, Chang Won Lee, Svetlana Stolpner, Chris Langley, Paul Grouchy, Steven Lake Waslander, Jonathan Kelly
AI summary
Problem
Vision-based anomaly detection for space robotics lacks domain-specific datasets and fails under extreme lighting and complex backgrounds. This gap hinders the autonomous inspection and hazard mitigation required for NASA's Lunar Gateway.
Approach
The authors generate a photorealistic synthetic dataset simulating lunar orbit conditions and propose a statistical algorithm that identifies anomalies by comparing live camera feeds against CAD-model-generated reference images.
Key results
- ALLO dataset: 51,409 photorealistic synthetic images with pixel-level anomaly annotations
- Benchmark reveals state-of-the-art industrial anomaly detectors severely underperform in space environments
- MRAD achieves 62.9% AP (pixel) and 75.0% AUROC (image) on the ALLO benchmark
- Dataset photorealism validated against real ISS imagery using LPIPS and FID metrics
Why it matters
Provides a critical benchmark and robust detection method to enable safe, autonomous robotic operations for future lunar and deep-space missions.
Abstract
NASA’s forthcoming Lunar Gateway space station, which will be uncrewed most of the time, will need to operate with an unprecedented level of autonomy. One key challenge is enabling the Canadarm3, the Gateway’s external robotic system, to detect hazards in its environment using its onboard inspection cameras. This task is complicated by the extreme and variable lighting conditions in space. In this paper, we introduce the visual anomaly detection and localization task for the space domain and establish a benchmark based on a synthetic dataset called ALLO (Anomaly Localization in Lunar Orbit). We show that state-of-the-art visual anomaly detection methods often fail in the space domain, motivating the need for new approaches. To address this, we propose MRAD (Model Reference Anomaly Detection), a statistical algorithm that leverages the known pose of the Canadarm3 and a CAD model of the Gateway to generate reference images of the expected scene appearance. Anomalies are then identified as deviations from this model-generated reference. On the ALLO dataset, MRAD surpasses state-of-the-art anomaly detection algorithms, achieving an AP score of 62.9% at the pixel level and an AUROC score of 75.0% at the image level. Given the low tolerance for risk in space operations and the lack of domain-specific data, we emphasize the need for novel, robust, and accurate anomaly detection methods to handle the challenging visual conditions found in lunar orbit and beyond.