To Harvest or Not to Harvest: Mapping and Decision-Making for a Selective Table Grape Harvesting Robot
Ruben Beumer, Leonardo Saraceni, Daniele Nardi, Duarte Antunes, René Molengraft van de, Thomas Ciarfuglia
AI summary
Problem
Selective harvesting robots struggle with sensor occlusions and data uncertainty, often defaulting to sub-optimal heuristics that compromise efficiency and crop quality. Real-time decision-making must account for complex reachability dependencies between clustered bunches.
Approach
The system fuses multi-view camera data using Kalman filtering to build a complete 3D quality map, then applies a dynamic programming algorithm on a reachability graph to sequentially optimize harvest quality and execution time.
Key results
- Multi-view Kalman filter pipeline for robust bunch tracking and 3D mapping
- Reachability graph modeling occlusion-induced access dependencies
- Dynamic programming algorithm for sequential quality and time optimization
- Real-time decision-making validated on a table grape harvesting robot
Why it matters
Provides a practical framework for agricultural robots to make optimal, quality-driven harvesting decisions in complex, occluded vineyard environments.
Abstract
This letter focuses on robotic harvesting of delicate crops such as table grapes, featuring selective harvesting based on individual product properties. The robot detects grape bunches and estimates their positions and quality attributes. However, sensor limitations and occlusions affect data completeness and accuracy, reducing the cost-effectiveness of automated harvesting systems. Determining in real-time the optimal harvesting order in the presence of uncertainty is therefore important for enhancing efficiency and grape quality for growers and consumers. This task is challenging not only due to data uncertainty, but also due to the need to consider factors such as obstructive low-quality bunches. Existing literature often resorts to sub-optimal approaches such as selecting the first available crop. In contrast, we propose (i) a mapping and tracking method based on multiple viewpoints to enhance bunch information quality and (ii) a decision-making algorithm in a decision-tree with a recursive structure based on a constructed reachability graph derived from the map to optimize harvested quality and execution time sequentially.