HEAPGrasp: Hand-Eye Active Perception to Grasp Objects with Diverse Optical Properties
Ginga Kennis, Shogo Arai
AI summary
Problem
Accurate 3-D measurement of transparent or specular objects remains difficult for conventional robotic systems, and moving a hand-eye camera through multiple viewpoints significantly increases handling execution time.
Approach
The method combines RGB semantic segmentation with Shape from Silhouette for robust 3-D reconstruction, guided by an active perception planner that optimizes camera trajectories to balance measurement accuracy and path length.
Key results
- 96.0% grasp success rate across diverse optical properties
- 52% reduction in hand-eye camera trajectory length
- 19% decrease in handling execution time versus baseline
- Robust semantic segmentation (mIoU 0.94) for transparent, specular, and opaque objects
Why it matters
Enables reliable, efficient robotic handling of challenging objects in real-world logistics and automation without requiring expensive specialized sensors.
Abstract
Autonomous robotic handling requires accurate 3- D scene measurement followed by grasp planning. Conventional systems struggle with transparent or specular objects. Addition- ally, in hand–eye setups, moving through multiple viewpoints increases handling execution time. In this paper, we propose HEAPGrasp—Hand-Eye Active Perception to Grasp objects with diverse optical properties. To measure such objects, we focus on the ability to segment objects regardless of their optical properties in RGB images. We employ Shape from Silhouette based on the segmented images for 3-D measurement. To shorten the time re- quired for multi-view capture with a hand-eye camera, we plan its trajectory using a cost function that balances 3-D measurement accuracy against its trajectory length. Real-robot experiments achieve a 96.0% grasp success rate on transparent, specular, and opaque objects, while reducing the hand-eye camera’s trajectory length by 52% and handling execution time by 19% relative to a baseline that circles around the scene for 3-D measurement.