Integrating Artificial Vision and Wearable Robotics: Adaptive Assistance Enabled by Manipulation Context Awareness
Sandro Ferrari, Emanuele Aimi, Francesco Missiroli, Federico Masiero, Maura Casadio, Lorenzo Masia
AI summary
Problem
Current industrial exoskeletons lack real-time, adaptable assistance tailored to specific manipulation tasks, often relying on impractical wearable sensors or failing to adjust to the objects workers handle.
Approach
The researchers integrated an RGB-D camera with a bimanual soft exoskeleton to run a vision pipeline that recognizes tools, tracks hands, and estimates object weight, dynamically modulating lifting assistance in real time.
Key results
- First fully vision-driven adaptive control for a bimanual soft upper-limb exoskeleton
- Vision pipeline achieves >93% object classification and >98% grasp detection accuracy
- Reduces biceps EMG activation by over 50% during industrial tool manipulation
- Robust to hand-object occlusions and camera repositioning on embedded hardware
Why it matters
This scalable, vision-only approach offers a practical solution for deploying ergonomic, context-aware exosuits in industrial settings to reduce worker fatigue and injury risk.
Abstract
Occupational exoskeletons are emerging as a promising solution for industrial applications, providing sup- port to reduce fatigue and the risk of musculoskeletal disorders. One of the main challenges limiting their widespread adoption is that most existing devices cannot deliver real-time, adaptable, and context-aware assistance. This paper presents the first fully vision-driven control strategy for a bimanual upper-limb soft exoskeleton, enabling adaptive assistance during industrial tool manipulation. The approach integrates three modules: tool recognition and segmentation, hand tracking with gesture recognition, and a fusion layer that ensures reliable under- standing of the manipulation context. This allows modulation of lifting assistance in real time according to the weight of the grasped object. Experiments with human participants demon- strated that the proposed approach reduces biceps activation by more than 50% compared to the no-support condition, while operating in real time on embedded hardware. The method is robust to hand–object occlusions, camera repositioning, and dynamic environments, demonstrating its practicality for industrial deployment. Overall, this work establishes vision- based control as a scalable solution for ergonomic, adaptive exoskeletons that enhance safety and productivity in demanding workplaces.