HITL-D: Human in the Loop Diffusion Assisted Shared Control
Riley Zilka, Sergey Khlynovskiy, Allie Wang, Martin Jagersand
AI summary
Problem
Operating high-degree-of-freedom assistive robotic arms via limited joystick interfaces imposes high mental workload and steep learning curves, limiting their practical adoption.
Approach
The framework splits control by keeping the human in charge of Cartesian position while a diffusion policy autonomously predicts and assists with end-effector orientation based on real-time scene data.
Key results
- Reduced average task completion time by 40% compared to traditional teleoperation
- Decreased perceived mental workload by 37% across multi-step manipulation tasks
- Improved user ratings for independence, intuitiveness, and confidence
- Achieved high success rates while requiring only a single expert demonstration per task for training
Why it matters
Lowers the operational barrier for assistive robotic systems, making them more accessible and less mentally taxing for users with motor impairments.
Abstract
Autonomous manipulation systems have achieved remarkable capabilities, yet the integration of human exper- tise with diffusion-based policies in shared control remains relatively unexplored. In this paper, we propose Human-In- The-Loop Diffusion (HITL-D), a shared control framework that enhances user performance in multi-step, insertion, and fine manipulation tasks. HITL-D leverages a novel combina- tion of diffusion-based policies and human control to provide autonomous end effector orientation updates conditioned on a scene point cloud and the Cartesian position of the end effector. This approach reduces the number of joystick control axes required, thereby lowering mental workload. In a multi-task user study with 12 participants, HITL-D reduced average task completion times by 40%, decreased perceived workload by 37%, and improved Likert-scale ratings for independence, intu- itiveness, and confidence compared to traditional teleoperation methods. These results demonstrate that HITL-D effectively integrates human expertise with autonomous assistance, im- proving both objective and subjective aspects of teleoperation.