Keypoint Semantic Integration for Improved Feature Matching in Outdoor Agricultural Environments
Rajitha de Silva, Jacob Swindell, Jonathan Cox, Marija Popovic, Cesar Cadena, Cyrill Stachniss, Riccardo Polvara
AI summary
Problem
Repetitive structures in outdoor agricultural environments like vineyards cause perceptual aliasing, leading to ambiguous keypoint descriptors and unreliable feature matching for robot navigation.
Approach
The authors propose Keypoint Semantic Integration (KSI), which enriches keypoint descriptors with semantic embeddings derived from instance segmentation masks, while leaving background keypoints unchanged.
Key results
- KSI improves keypoint matching accuracy across multiple months and descriptor types
- Semantic enrichment reduces relative and absolute pose errors during navigation
- Cross-seasonal visual localization accuracy increases by over 35% in feature-scarce rows
- Introduction of the SemanticBLT dataset with panoptic segmentation annotations for vineyards
Why it matters
Enables more reliable vision-based navigation for agricultural robots operating in visually repetitive, seasonally changing outdoor environments.
Abstract
Robust robot navigation in outdoor environments requires accurate perception systems capable of handling vi- sual challenges such as repetitive structures and changing appearances. Visual feature matching is crucial to vision-based pipelines but remains particularly challenging in natural out- door settings due to perceptual aliasing. We address this issue in vineyards, where repetitive vine trunks and other natural elements generate ambiguous descriptors that hinder reliable feature matching. We hypothesise that semantic information tied to keypoint positions can alleviate perceptual aliasing by enhancing keypoint descriptor distinctiveness. To this end, we introduce a keypoint semantic integration technique that improves the descriptors in semantically meaningful regions within the image, enabling more accurate differentiation even among visually similar local features. We validate this approach in two vineyard perception tasks: (i) relative pose estimation and (ii) visual localisation. Our method improves matching accuracy across all tested keypoint types and descriptors, demonstrating its effectiveness over multiple months in chal- lenging vineyard conditions.