Bi-Hap: A Bi-Directional Learning-Based Control and Momentum-Based Haptic Feedback System for Dexterous In-Hand Telemanipulation
Haoyang Wang, Haoran Guo, Zhengxiong Li, Lingfeng Tao
AI summary
Problem
Current telemanipulation frameworks often treat control and feedback in isolation, lacking bidirectional information flow and realistic torque perception for dexterous in-hand tasks.
Approach
Bi-Hap uses an IMU to capture operator motion for a deep reinforcement learning controller, paired with a lightweight, palm-sized momentum-actuated device that dynamically modulates torque and vibration feedback based on real-time task error.
Key results
- First integrated system coupling learning-based control with ungrounded momentum-based haptic feedback
- DRL policy bridging human partial observations with robust robot execution
- Palm-sized torque and vibration feedback device with error-adaptive modulation
- Experimental validation showing <0.03s latency, <0.01Nm torque RMSE, and improved manipulation accuracy
Why it matters
Enables more intuitive and precise remote manipulation for applications in telerehabilitation, human-robot collaboration, and complex teleoperation.
Abstract
Dexterous in-hand telemanipulation demands precise control and realistic haptic feedback to achieve stable and intuitive human–robot interaction. Existing systems often emphasize isolated control policies or unidirectional force feedback, limiting performance in tasks that require coordinated bidirectional information flow. In this work, we introduce Bi-Hap, a bi-directional learning-based control and momentum-based haptic feedback system for real-time, in-hand telemanipulation. On the control side, Bi-Hap leverages an inertial measurement unit to capture operator motion and drives a deep reinforcement learning policy that enables robust and adaptive manipulation of objects with fine rotational dexterity. On the feedback side, a compact, palm-sized momentum-actuated mechanism delivers torque and vibration cues directly to the operator, augmented by an error-adaptive strategy that modulates feedback intensity based on task states. When integrated, this closed-loop design establishes an immersive bidirectional control–feedback framework. Experimental results show that Bi-Hap achieves low feedback latency (<0.03s), high torque fidelity (RMSE <0.01Nm), and significantly improved telemanipulation performance by elevating manipulation accuracy, responsiveness, and operator situational awareness in diverse task settings.