Gaze-Based Teleoperation with Intent Inference Model for Robotic Manipulators
Yanjia Yuan, Chong Peng, Dihui Chu, Qianqian WANG, Qiang Gao, Yunlong Tang, Xiaoyu Wang
AI summary
Problem
Traditional gaze-based robotic control relies on rigid fixation thresholds that trigger pre-programmed actions, lacking adaptability in unstructured environments. Direct gaze control also suffers from involuntary eye fluctuations that cause oscillatory, inefficient robot motion.
Approach
The system trains a Gaussian Mixture Regression model on synchronized gaze and haptic teleoperation data to learn a nonlinear mapping from eye movements to intended robot trajectories. Real-time gaze inputs are continuously processed by this model to generate smooth, predictive end-effector commands without prior environmental knowledge.
Key results
- Statistically significant improvement over G-HMM in efficiency and smoothness
- Mitigation of involuntary eye fluctuations reducing motion oscillations
- Enhanced user sense of involvement, control, and perceived speed
- Real-time gaze-to-motion translation without pre-programmed paths or environmental mapping
Why it matters
Provides a practical, non-invasive interface for assistive and collaborative robotics that reduces cognitive load and adapts to dynamic tasks without complex environmental modeling.
Abstract
Eye gaze-based control interfaces provide a non- invasive means of enhancing human-robot collaboration for activities of daily living and can reduce the cognitive burden on operators performing complex tasks. Eye gaze has traditionally been used for "gaze triggering," where fixating on an object activates pre-programmed robotic movements. In this work, we propose a gaze-based robotic teleoperation approach that utilizes real-time gaze data to guide the freeform movement of robotic manipulators. The proposed approach incorporates a Gaussian Mixture Regression (GMR)-based intent inference model to capture the nonlinear relationship between gaze data and the operator’s intended robotic movements. For bench- marking, we further implemented a Gaussian Hidden Markov Model (G-HMM) to provide a comparable probabilistic frame- work for intent inference. Experimental results demonstrate that the GMR-based approach achieves a statistically significant improvement over G-HMM in terms of control efficiency, trajectory smoothness against involuntary eye fluctuations, as well as enhancing the user’s sense of involvement and control.