ActiveSPN: Active Soft Polyhedral Networks with Pose Estimation for In-Finger Object Manipulation
Sen Li, Chengxiao Dong, Chaoyang Song, Fang Wan
AI summary
Problem
Integrating reliable perception into continuously moving active surfaces is challenging, and existing soft grippers often lack the dexterity needed for complex in-hand manipulation tasks.
Approach
The authors developed a soft polyhedral finger covered by a transparent active belt, paired with a built-in camera and a semi-supervised variational autoencoder to estimate object pose directly from in-finger vision.
Key results
- Demonstrated multi-degree-of-freedom in-finger manipulation including two-axis rotation and one-axis translation
- Achieved real-time pose estimation with mean translational error of 2.59 mm and rotational error of 7°
- Developed a semi-supervised VAE pipeline for robust in-hand pose estimation using built-in cameras
- Enabled closed-loop control for consistent dexterous manipulation across diverse objects
Why it matters
Offers a compact, vision-driven solution for dexterous robotic manipulation that bypasses traditional wiring and sensor limitations in active soft grippers.
Abstract
Robotic grippers aim to replicate the remarkable functionalities of the human hand by providing advanced per- ception, adaptability, stability, and dexterity for complex tasks. Achieving these capabilities demands a sophisticated design hi- erarchy and robust perception mechanisms that ensure accurate manipulation. This letter introduces Active Soft Polyhedral Net- works (ActiveSPN), a gripper design that leverages an active, non-biomimetic surface for precise in-hand manipulation. A vision system integrated directly into the fingers further facilitates accu- rate pose estimation of the in-finger object. The proposed system includes: (i) a soft polyhedral network featuring a transparent active belt to deliver complete three-dimensional adaptation and dexterous in-finger motion, and (ii) a generative learning-based pipeline for in-finger pose estimation. Experimental results demon- strate the ability of ActiveSPN to execute multi-degree-of-freedom in-finger manipulations, including two-axis rotation and one-axis translation. Moreover, the integrated vision-based pose estima- tion provides robust, real-time predictions, supporting consistent closed-loop control. Across diverse objects, the system achieves mean translational errors of 2.59 mm and rotational errors of 7◦, highlighting a promising paradigm for compact, efficient, and dexterous robotic manipulation.