Dynamic Bimanual Human-To-Robot Object Handovers Using Motion Prediction Deep Neural Networks

Facilitating dynamic bimanual handovers between humans and robots is a complex endeavor that requires inte- grating human pose estimation, motion prediction, and gener- ating appropriate trajectories for receiving robots. This study introduces a method designed to predict required bimanual receiver robot motion during handover tasks, leveraging pose estimation and motion prediction networks. Additionally, we propose a real-time control approach for a dual-arm humanoid robot to dynamically adjust its receiving trajectories. We evaluate the ability of neural networks to accurately predict receiver trajectories and thus improve the handover process. We compare long short-term memory (LSTM) and transformer architectures for motion prediction and also assess prediction accuracy of both absolute and relative receiving trajectories as well trajectories for each separate robot arm, and show that the use of absolute and relative coordinates is beneficial for generating more accurate receiver motions.