Robotic Grasping and Placement Controlled by EEG-Based Hybrid Visual and Motor Imagery
Yichang Liu, Tianyu Wang, Ziyi Ye, Yawei Li, Yu-Gang Jiang, Shouyan Wang, Yanwei Fu
AI summary
Problem
Current BCI-robot systems rely on constrained explicit cues or simple commands, failing to bridge high-level human cognitive intent with complex, intuitive robotic actions in dynamic environments.
Approach
The framework trains offline decoders for visual imagery (object selection) and motor imagery (placement pose), then deploys them in a zero-shot online pipeline to control a robotic arm via a cue-free EEG protocol.
Key results
- Offline decoding accuracies of 44.11% (visual imagery) and 76.53% (motor imagery)
- Online decoding accuracies of 40.23% (visual imagery) and 62.59% (motor imagery)
- End-to-end pick-and-place task success rate of 20.88%
- Robust performance in occluded target and supine posture scenarios
Why it matters
Demonstrates the feasibility of cue-free, high-level cognitive BCI control for practical human-robot collaboration and real-world manipulation tasks.
Abstract
We present a framework that integrates EEG- based visual and motor imagery (VI/MI) with robotic control to enable real-time, intention-driven grasping and placement. Motivated by the promise of BCI-driven robotics to enhance human-robot interaction, this system bridges neural signals with physical control by deploying offline-pretrained decoders in a zero-shot manner within an online streaming pipeline. This establishes a dual-channel intent interface that translates visual intent into robotic actions, with VI identifying objects for grasping and MI determining placement poses, enabling intuitive control over both what to grasp and where to place. The system operates solely on EEG via a cue-free imagery protocol, achieving integration and online validation. Imple- mented on a Base robotic platform and evaluated across diverse scenarios, including occluded targets or varying participant postures, the system achieves online decoding accuracies of 40.23% (VI) and 62.59% (MI), with an end-to-end task success rate of 20.88%. These results demonstrate that high-level visual cognition can be decoded in real time and translated into executable robot commands, bridging the gap between neural signals and physical interaction, and validating the flexibility of a purely imagery-based BCI paradigm for practical human–robot collaboration.