A Whole-Body Integrated AVATAR System: Implementation of Telepresence with Intuitive Control and Immersive Feedback

This paper proposes an intuitive and immersive whole-body teleoperation system with motion-based control and multi-modal feedback. The system consists of an anthropomorphic teleoperated robot and a haptic interface platform. The teleoperated robot has dual arms with dexterous hands, a head with a neck, a waist, giving it a human-like appearance and a large range of motion (ROM), as well as an omnidirectional mobile platform for improved mobility. The haptic interface platform enables a human operator to control the robot intuitively by measuring the operator's motion with a motion capture system, providing haptic feedback to the user's arms, fingers, and feet, and providing 3D image feedback. Additionally, facial animation further enhances immersion by synchronizing the face expression of the robot with the user’s voice. The proposed teleoperation system offers a promising solution for the human-oriented robotic avatar system, which was verified through a global competition, the $10M ANA Avatar XPRIZE. The system was successfully evaluated with 45 minutes of training time for users who were new to our system. And the lessons learned from the competition and future improvements are discussed.