Stability and Transparency in Mixed Reality Bilateral Human Teleoperation
David Gregory Black, Septimiu E. Salcudean
AI summary
Problem
The stability and transparency of mixed reality human teleoperation remain unexplored, despite its potential for low-cost telemedicine. Time delays and human tracking limitations threaten system stability and performance.
Approach
The authors developed a linear mathematical model of the human-in-the-loop teleoperation system and analyzed various control architectures to evaluate stability and transparency under different time delays.
Key results
- Derived a hybrid matrix model capturing novice tracking dynamics and time delays.
- Demonstrated that system instability severely degrades usability but is non-destructive.
- Achieved stable, transparent operation with delays under 200 ms using 3-channel teleoperation.
- Enabled stable operation with large delays via model-mediated control with local pose and force feedback.
Why it matters
Provides foundational control strategies for safe, effective mixed reality teleguidance in telemedicine and remote assistance applications.
Abstract
Recent work introduced the concept of human teleoperation (HT), where the remote robot typically considered in conventional bilateral teleoperation is replaced by a novice person wearing a mixed reality head-mounted display and tracking the motion of a virtual tool controlled by an expert. HT has advantages in cost, complexity, and patient acceptance for telemedicine in low-resource communities or remote loca- tions. However, the stability, transparency, and performance of bilateral HT are unexplored. In this paper, we therefore develop a mathematical model of the HT system using test data. We then analyze various control architectures with this model and implement them with the HT system, testing volunteer operators and a virtual fixture-based simulated patient to find the achievable performance, investigate stability, and determine the most promising teleoperation scheme in the presence of time delays. We show that instability in HT, while not destructive or dangerous, makes the system impossible to use. However, stable and transparent teleoperation are possible with small time delays (< 200 ms) through 3-channel teleoperation, or with large time delays through model-mediated teleoperation with local pose and force feedback for the novice.