Do You Know the Way? Human-In-The-Loop Understanding for Fast Traversability Estimation in Mobile Robotics
Andre Schreiber, Katherine Driggs-Campbell
AI summary
Problem
Existing vision-based traversability methods either require tedious manual labeling, rely on noisy self-supervised robot experience, or fail to adapt to new environments during deployment.
Approach
The system uses a DINOv2 foundation model to predict traversability, automatically flags unfamiliar scenes, prompts a human for sparse relative annotations, and rapidly retrains the model from scratch on the updated data.
Key results
- Sparse human annotations enable rapid model updates without teleoperation
- Novelty detection selectively triggers human labeling only for unfamiliar scenes
- Foundation model features with upsampling yield state-of-the-art prediction accuracy
- High-fidelity simulator successfully bridges the gap to real-world traversability prediction
Why it matters
Allows field robots to safely navigate unpredictable terrains by continuously learning from minimal human input without risking hardware damage.
Abstract
The increasing use of robots in unstructured envi- ronments necessitates the development of effective perception and navigation strategies to enable field robots to successfully perform their tasks. In particular, it is key for such robots to understand where in their environment they can and cannot travel—a task known as traversability estimation. However, existing geometric approachestotraversabilityestimationmayfailtocapturenuanced representations of traversability, whereas vision-based approaches typically either involve manually annotating a large number of images or require robot experience. In addition, existing methods can struggle to address domain shifts as they typically do not learn during deployment. To this end, we propose a human-in-the-loop (HiL) method for traversability estimation that prompts a human for annotations as-needed. Our method uses a foundation model to enable rapid learning on new annotations and to provide accurate predictions even when trained on a small number of quickly- provided HiL annotations. We extensively validate our method in simulation and on real-world data, and demonstrate that it can provide state-of-the-art traversability prediction performance.