Image-Based Roadmaps for Vision-Only Planning and Control of Robotic Manipulators
Sreejani Chatterjee, Abhinav Gandhi, Berk Calli, Constantinos Chamzas
AI summary
Problem
Vision-based control techniques lack collision-free path planning capabilities that operate without explicit robot models or proprioceptive sensors, limiting their use on complex or inexpensive manipulators.
Approach
The authors construct a probabilistic roadmap directly in image space using tracked visual keypoints on the robot’s body, adapting sampling, nearest-neighbor search, and collision checking to function without geometric models. They evaluate two distance metrics for the roadmap: a neural network that estimates joint displacements and a direct Euclidean distance in image space.
Key results
- First roadmap constructed entirely in image space using natural visual keypoints without robot models
- Learned-distance roadmap achieved 100% control convergence success in experiments
- Predefined image-space distance roadmap enabled faster transient responses but lower convergence reliability
- Validated real-world collision avoidance and path tracking on a robotic manipulator using only vision feedback
Why it matters
Enables reliable, model-free motion planning and control for soft, underactuated, or inexpensive robots where traditional geometric modeling and proprioceptive sensing are impractical.
Abstract
This work presents a motion planning framework for robotic manipulators that computes collision-free paths directly in image space. The generated paths can then be tracked using vision-based control, eliminating the need for an explicit robot model or proprioceptive sensing. At the core of our approach is the construction of a roadmap entirely in image space. To achieve this, we explicitly define sampling, nearest-neighbor selection, and collision checking based on visual features rather than geometric models. We first collect a set of image space samples by moving the robot within its workspace, capturing keypoints along its body at different configurations. These samples serve as nodes in the roadmap, which we construct using either learned or predefined distance metrics. At runtime, the roadmap generates collision- free paths directly in image space, removing the need for a robot model or joint encoders. We validate our approach through an experimental study in which a robotic arm follows planned paths using an adaptive vision-based control scheme to avoid obstacles. The results show that paths generated with the learned-distance roadmap achieved 100% success in control convergence, whereas the predefined image space distance roadmap enabled faster transient responses but had a lower success rate in convergence.