Gaze-Guided Robotic Vascular Ultrasound Leveraging Human Intention Estimation
Yuan Bi, Yang Su, Nassir Navab, Zhongliang Jiang
AI summary
Problem
Traditional ultrasound exams suffer from operator variability, while existing robotic systems struggle to navigate vascular bifurcations and maintain optimal probe contact on curved surfaces without explicit human guidance.
Approach
The system processes raw gaze signals through a human intention estimation module to generate stabilized attention heatmaps, which guide a transformer-based segmentation network and control the robot to track target vessels and adjust probe orientation.
Key results
- Gaze-guided segmentation network improves vessel segmentation accuracy using stabilized attention heatmaps
- Human intention estimation module successfully filters noisy gaze signals to predict operator focus
- Confidence-driven orientation correction optimizes linear probe contact on uneven surfaces
- System validated on a realistic arm phantom, demonstrating dynamic vessel tracking and hands-free operation
Why it matters
Enables hands-free, reproducible vascular ultrasound scanning for surgeons in intra-operative settings where manual probe control is impractical.
Abstract
Medical ultrasound has been widely used to examine vascular structure in modern clinical practice. However, tra- ditional ultrasound examination often faces challenges related to inter- and intra-operator variation. The robotic ultrasound system (RUSS) appears as a potential solution for such challenges because of its superiority in stability and reproducibility. Given the complex anatomy of human vasculature, multiple vessels often appear in ultrasound images, or a single vessel bifurcates into branches, complicating the examination process. To tackle this challenge, this work presents a gaze-guided RUSS for vascular applications. A gaze tracker captures the eye movements of the operator. The extracted gaze signal guides the RUSS to follow the correct vessel when it bifurcates. Additionally, a gaze-guided segmentation network is proposed to enhance segmentation robustness by exploiting gaze information. However, gaze signals are often noisy, requiring interpretation to accurately discern the operator’s true intentions. To this end, this study proposes a stabilization module to process raw gaze data. The inferred attention heatmap is utilized as a region proposal to aid segmen- tation and serve as a trigger signal when the operator needs to adjust the scanning target, such as when a bifurcation appears. To ensure appropriate contact between the probe and surface during scanning, an automatic ultrasound confidence-based orientation correction method is developed. In experiments, we demonstrated the efficiency of the proposed gaze-guided segmentation pipeline by comparing it with other methods. Besides, the performance of the proposed gaze-guided RUSS was also validated as a whole on a realistic arm phantom with an uneven surface.