A Spatiotemporal Brain Activity Visualization and Assessment Framework for Human-Robot Cognitive Interaction Training
Zonghai Huang, Lianchi Zhang, Jingting Zhang, Fengjun Mu, Rui Huang, Hong Cheng
AI summary
Problem
Existing methods struggle to capture the complex, nonlinear spatiotemporal dynamics of brain activity during human-robot interaction, limiting accurate online assessment and adaptive modulation for closed-loop brain training.
Approach
The authors combine deterministic learning with neural population theory to extract spatiotemporal features from EEG signals, enabling real-time brain state visualization and adaptive adjustment of immersive VR training parameters.
Key results
- Improved brain activity assessment accuracy by 8.86%
- Real-time visualization of nonlinear spatiotemporal brain dynamics
- Adaptive VR task parameter modulation based on assessed brain states
- Enhanced cognitive control and training outcomes in experiments
Why it matters
Provides a reliable, interpretable method for real-time neural state tracking, advancing personalized and adaptive rehabilitation and brain training applications.
Abstract
Accurately assessing brain activity to modulate training parameters online is crucial for improving the human- robot cognitive interaction (HRCI) performance in closed-loop brain training. The major challenge for this technique lies in how to accurately model and characterize the intrinsic behavior of brain activity in HRCI process, which typically exhibits a dynamic manner across spatial and temporal scales. In this study, we propose a dynamic perspective to visualize the spatiotemporal evolution of brain activity during HRCI process, thus enabling assessment of brain states and adaptive modu- lation during rehabilitation. A novel framework is developed to model the spatiotemporal dynamics of brain activity by in- tegrating deterministic learning with neural population theory. It demonstrates a remarkable capability to mine and visualize the complex nonlinear dynamics of brain activity, encompassing both temporal evolution and spatial connectivity patterns. The proposed model not only visualizes of spatiotemporal brain dynamics but also enables online assessment of brain states, which can facilitate optimal modulation of HRCI process and improve the brain training efficiency. The method is validated using a panoramic virtual reality system. Results show that our method improves the accuracy of brain activity assessment by 8.86%, effectively demonstrating that it accurately visualizes spatiotemporal brain dynamics and enhances training outcomes when integrated with HRCI.