A Cable-Driven Soft Robotic Hand with an In-Hand RGB-D Camera for Dexterous Grasping and Manipulation
Zhanfeng Zhou, Runze Zuo, Matthew Du, Shaojia Wang, Sebastian Levy, Yu Sun, Xinyu Liu
AI summary
Problem
Existing soft robotic hands lack active, bidirectional multidegree-of-freedom motion in each finger and effective in-hand visual feedback, limiting their dexterity and manipulation capabilities.
Approach
The researchers developed a five-finger cable-driven soft hand where each finger independently performs flexion/extension and bidirectional adduction/abduction, integrated with a palm-mounted RGB-D camera for real-time visual feedback.
Key results
- Five-finger design with independent bidirectional lateral and bending motion
- Palm-integrated RGB-D camera providing unobstructed in-hand vision
- Vision-guided grasping strategy with real-time slip detection and compensation
- Hierarchical visually servoed controller enabling closed-loop in-hand manipulation
Why it matters
Provides a highly dexterous, visually guided platform for safe object handling in household tasks, food sorting, and prosthetic applications.
Abstract
The aspiration to replicate the capabilities of the human hand has driven innovations in the design of soft robotic hands. Despite these advancements, many existing designs of soft hands still lack effective in-hand vision and the ability for each finger to achieve active multidegree-of-freedom motion. This article proposes a cable-driven soft robotic hand that can achieve dex- terous grasping and manipulation, vision-guided grasping, vision- based slip detection and compensation, as well as visually servoed in-hand manipulation. The hand has five soft fingers, each ca- pable of independent flexion/extension motion and bidirectional ad/abduction motion. A red–green–blue-depth (RGB-D) camera is integrated into the palm of the soft hand to enable in-hand vision capability. Modeling of the soft hand is established to analyze its kinematics, statics, and manipulability. A series of experiments are conducted to demonstrate its dexterous grasping and manipulation capabilities on a variety of objects. Using 3-D point cloud data from the in-palm camera, an effective vision-guided grasping strat- egy is developed to grasp objects on a table. The in-hand vision also enables slip detection and compensation during grasping to maintain the grasp stability. Furthermore, a hierarchical, visually servoed controller is developed to perform closed-loop in-hand object manipulation. With its high dexterity and visual feedback capabilities, the soft hand will find important applications such as household object manipulation and food picking/sorting, and may also be used as a prosthetic hand or an auxiliary hand for humans.