Beyond Waypoints: Semantic-Centric Autonomy with Unreliable Maps through Learned Abstractions
Akila Saravanan, Songyuan Zhang, Travis Manderson, Nicholas Roy, Chuchu Fan
AI summary
Problem
Autonomous navigation typically fails when overhead maps are stale, misaligned, or noisy, as coordinate-based waypoint following breaks down. This paper addresses how to execute reliable high-level plans in unstructured environments without precise metric localization.
Approach
The method transforms raw LiDAR scans into low-dimensional embeddings that are clustered into human-interpretable behavioral states, enabling navigation via a topological sequence of these semantic regions guided by a general bearing rather than exact coordinates.
Key results
- Learned behavioral vocabulary from LiDAR via convolutional VAE and HDBSCAN
- Navigation framework planning via topological sequences of semantic clusters
- 53 and 55 percentage point higher mission success under map distortions
- Zero-shot policy transfer from 2D simulation to CARLA and real-world
Why it matters
Enables robust autonomous navigation for disaster relief and exploration where traditional metric maps are unreliable or unavailable.
Abstract
Autonomous navigation that relies on precise met- ric maps is inherently fragile to environmental changes and mapping inaccuracies. These discrepancies often lead to failures in localization and path planning, as the robot’s internal representation of the world no longer matches reality. We propose an alternative navigation approach that instead focuses on how a robot interacts with its surroundings rather than its precise metric position. Our core contribution is a learned behavioral vocabulary conditioned on raw sensor data that can be used to compose plans for navigation. Our system transforms LiDAR data into low-dimensional learned embeddings which are clustered to create a set of abstract, human-interpretable behaviors (e.g., along wall, exiting intersection, on bridge). This representation allows the robot to control its behavior with respect to the embedding rather than controlling its state with respect to a specific metric cost function or waypoint, thereby minimizing the impact of map and position inaccu- racies. We define the mission as a topological sequence of behavioral clusters on the overhead map, enabling high-level navigation. This approach provides a robust way to decompose the environment into recognizable and actionable states that can reliably compose a plan, even on stale maps with environmental deformations and world changes. Our method achieves higher navigation success under intentional map distortions, with av- erage mission success rates 53 and 55 percentage points higher for short and long term plans respectively when compared to baselines which rely on accurate metric maps.