NEUSIS: A Compositional Neuro-Symbolic Framework for Autonomous Perception, Reasoning, and Planning in Complex UAV Search Missions
Zhixi Cai, Cristian Rojas Cardenas, Kevin Leo, Chenyuan Zhang, Kal Backman, Hanbing Li, Boying Li, Mahsa Ghorbanali, Stavya Datta, Lizhen Qu, Julian Gutierrez, Alexey Ignatiev, Yuan-Fang Li, Mor Vered, Peter Stuckey, Maria Garcia de la Banda, Hamid Rezatofighi
AI summary
Problem
Autonomous UAVs struggle to reliably search for specific targets in large, dynamic, and hazard-prone environments due to the computational demands of monolithic models and the lack of explicit world modeling and visual reasoning in existing compositional systems.
Approach
NEUSIS integrates a neuro-symbolic perception module for 3D visual grounding, a probabilistic world model for dynamic belief updating, and a hierarchical symbolic planner to process sensor data, maintain an interpretable environment map, and generate efficient search paths.
Key results
- GRiD enables robust 3D visual grounding via multimodal vision models
- Probabilistic world model refines noisy sensor data with Bayesian filtering
- Hierarchical planner SNaC optimizes AOI selection and obstacle-aware coverage
- NEUSIS outperforms baselines in localization accuracy and navigation efficiency
Why it matters
It provides a robust, interpretable, and efficient architecture for autonomous UAV search and rescue operations in complex, dynamic environments where safety and reliability are critical.
Abstract
This paper addresses the problem of autonomous UAV search missions, where a UAV must locate specific Entities of Interest (EOIs) within a time limit, based on brief descriptions in large, hazard-prone environments with keep-out zones. The UAV must perceive, reason, and make decisions with limited and un- certain information. We propose NEUSIS, a compositional neuro- symbolic system designed for effective UAV search and navigation in realistic scenarios. NEUSIS integrates neuro-symbolic visual perception, reasoning, and grounding (GRiD) to process raw sensory inputs, maintains a probabilistic world model for environ- ment representation, and uses a hierarchical planning component (SNaC) for efficient path planning. Experimental results from simulated urban search missions using AirSim and Unreal Engine show that NEUSIS outperforms state-of-the-art baselines for both perception and planning. These results demonstrate the effectiveness of our compositional neuro-symbolic approach in handling complex scenarios, making it a promising solution for autonomous UAV systems in search missions.