Scene-Aware Robotic Light Pipe Control for Vitreoretinal Surgery
Wenjun Lin, Wending Zhang, Chin-Boon Chng, Yong Jun Tan, Chee Kong Chui
AI summary
Problem
Vitreoretinal surgery currently requires surgeons to manually control a light pipe with their non-dominant hand, increasing physical and cognitive load while limiting the feasibility of complex multi-arm robotic procedures.
Approach
The authors propose a vision-based robotic assistant that uses a novel detect-and-match neural network to identify surgical tools, tissues, and activities, feeding this data into an optimization algorithm to automatically adjust the light pipe's position.
Key results
- DMNet achieves 63.89% mAP for surgical activity detection and 99.29% AP50 for object detection
- Optimization-based control achieves a 2.5 mm average point-reaching error in phantom experiments
- Automated light positioning successfully tracks surgical targets in real-time
- System enables automatic documentation and tool trajectory recording for skill assessment
Why it matters
It alleviates surgeon fatigue and cognitive distraction while improving procedural safety, enabling more complex multi-arm vitreoretinal surgeries.
Abstract
Surgical robotics have revolutionized medical pro- cedures by offering enhanced precision and reduced compli- cations. However, vitreoretinal surgery still relies heavily on manual techniques, where surgeons manage both a surgical tool and a light pipe, complicating operations and potentially affecting outcomes. To improve efficiency and outcomes while reducing workloads on surgeons, a novel vision-based robot- assisted system with advanced surgical scene understanding ability is proposed. The system automatically positions a light pipe held by a specialized surgical robot through optimization- based visual collaborative control. By identifying target areas for automatic illumination, the system allows surgeons to focus on surgical tasks and supports more complex surgeries such as three-arm procedures. Besides, the system enhances surgical safety by detecting surgical activities and dangerous areas and issuing alerts accordingly. Postoperatively, the system records tool trajectories and detected activities, providing data for surgical reports, skill evaluation, and training. Experiments prove the effectiveness of the control system, visual algorithm, and overall collaborative system.