Exploring Haptic Augmentation and Language Design for Smartphone-Based Teleoperation
Zachary Andrew Christopher Baylis, Ziling Chen, Rolando Bautista Montesano, Yeo Jung Yoon, Thomas Bohné, Slawomir Konrad Tadeja, John Liu
AI summary
Problem
Smartphone-based teleoperation interfaces currently rely almost exclusively on visual feedback, which increases operator cognitive load and limits precision in manufacturing tasks.
Approach
The researchers implemented five symbolic vibration cues on an unmodified iPhone to guide a robotic arm during sorting and insertion tasks, comparing performance between visual-only and visual-plus-haptic conditions in a controlled user study.
Key results
- 42% reduction in total errors and lower perceived workload
- 94% and 81% recognition rates for continuous alignment and error boundary cues
- Lower reliability for brief contact and gripper state cues
- Design guidelines prioritizing simple, continuous, and intense haptic signals
Why it matters
Provides actionable design guidelines for integrating low-cost, symbolic haptic feedback into scalable smartphone-driven industrial teleoperation systems.
Abstract
Smartphone-based teleoperation is gaining trac- tion as a versatile remote control solution, using widely available hardware to provide a portable and scalable interface for teler- obotics. However, a crucial limitation of such an approach is the lack of effective haptic feedback, which restricts accuracy and increases operator workload. While smartphones offer a low- entry barrier as well as both portability and scalability, current interfaces rely almost exclusively on visual cues. To address this gap, we investigate the use of symbolic haptic feedback delivered through an unmodified mobile device to support re- mote manipulation tasks. We designed a combined teleoperation task that integrates object sorting and peg-in-hole insertion, embedding five candidate haptic cues (i.e., contact, gripper state, alignment, error boundary, and motion initiation). A within-subjects study with 16 participants compared visual- only and visual-plus-haptic conditions. Results show that haptic augmentation reduced total errors by 42% and significantly lowered perceived workload. Continuous cues for alignment and error boundaries achieved the highest recognition rates of 94% and 81%, respectively, while brief state cues were less reliably interpreted. Post-task interviews highlighted user preference for simple, continuous, and intense signals in visually ambiguous scenarios. Our findings provide new design guidelines for haptic cue prioritisation and encoding strategies.