Neural Profiling with fNIRS of Operator Performance in Teleoperated Human-Like Social Robot Interactions
David Achanccaray, Javier Andreu-Perez, Hidenobu Sumioka
AI summary
Problem
Teleoperating social robots demands complex cognitive skills, yet objective methods to profile operator performance and skill acquisition remain underexplored. This gap hinders the development of adaptive training protocols and performance-aware interfaces.
Approach
Researchers recorded prefrontal cortex hemodynamic responses and functional connectivity using mobile fNIRS while 32 participants teleoperated a humanoid robot in a simulated clinic, then grouped operators by performance using behavioral and neural metrics.
Key results
- Significant differences in hemoglobin oxygenation curve width and multiscale entropy between performance groups
- Distinct prefrontal cortex functional connectivity patterns and leaf fraction metrics separate operator skill levels
- Gaussian mixture clustering successfully classifies high- and low-performing operators using combined neural and behavioral data
- Neural profiles correlate with task efficiency, workload, and presence metrics
Why it matters
These neural biomarkers enable objective skill assessment, paving the way for adaptive teleoperation training and real-time performance monitoring in human-robot interaction.
Abstract
Social robot teleoperation is a skill that must be ac- quired through practice with the social robot. Mobile neuroimaging and human-computer interface performance metrics permit the gathering of information from the operators’ systemic and behav- ioral responses associated with their skill acquisition. Profiling the skill levels of social robot operators using this information can help improve training protocols. In this study, thirty-two participants performed real-world social robot teleoperation tasks. Brain func- tion signals from the prefrontal cortex (PFC), and behavioral data from interactions with the system were collected using functional near-infrared spectroscopy (fNIRS). Participants were divided into two groups (high and low performance) based on an integrative metric of task efficiency, workload, and presence when operating the social robot. Significant differences were found in the operation time, width, and multiscale entropy of the hemoglobin oxygenation curve of the operator’s PFC. Functional connectivity in the PFC also depicted differences in the low- and high-performance groups when connectivity networks were compared and in the leaf fraction metrics of the functional networks. These findings contribute to un- derstanding the operator’s progress during teleoperation training protocols and designing the interface to assist in enhancing task performance.