Point Cloud-Based Grasping for Soft Hand Exoskeleton
Chen Hu, Enrica Tricomi, Eojin Rho, Daekyum Kim, Lorenzo Masia, SHAN LUO, Letizia Gionfrida
AI summary
Problem
Controlling soft hand exoskeletons to assist users with grasping impairments remains difficult due to the complexity of environmental understanding, while existing data-driven vision methods require extensive labeled datasets, struggle with generalizability, and impose high computational costs for real-time wearable use.
Approach
The system uses a wrist-mounted depth camera to reconstruct 3D point clouds, identifies objects and grasp targets through geometric modeling and clustering, and automatically triggers a PID controller to actuate the exoskeleton when a target object is within range.
Key results
- Achieved a state-of-the-art Grasping Ability Score of 91 ± 2% across 15 diverse objects
- Maintained successful reconstruction and grasping for unseen objects, demonstrating strong generalizability
- Eliminated reliance on extensive labeled datasets by leveraging geometric point cloud modeling over deep learning
- Enabled real-time, computationally efficient vision-based control suitable for wearable robotic assistance
Why it matters
This approach advances wearable assistive robotics by providing a computationally efficient, dataset-free control method that can reliably help individuals with hand impairments perform daily grasping tasks.
Abstract
Grasping is a fundamental skill for interacting with and manipulating objects in the environment. However, this ability can be challenging for individuals with hand impair- ments. Soft hand exoskeletons designed to assist grasping can enhance or restore essential hand functions, yet controlling these soft exoskeletons to support users effectively remains difficult due to the complexity of understanding the environment. This study presents a vision-based predictive control framework that leverages contextual awareness from depth perception to predict the grasping target and determine the next control state for activation. Unlike data-driven approaches that require extensive labelled datasets and struggle with generalizability, our method is grounded in geometric modelling, enabling robust adaptation across diverse grasping scenarios. The Grasping Ability Score (GAS) was used to evaluate performance, with our system achieving a state-of-the-art GAS of 91 ± 2% across 15 objects and healthy participants, demonstrating its effectiveness across different object types. The proposed approach maintained recon- struction success for unseen objects, underscoring its enhanced generalizability compared to learning-based models.