Overcoming Imperfect Kinematics in Surgical Robotics through Sim-To-Real Visuomotor Learning
Zhaoxuan Yan, Kaizhong Deng, Zhaoyang Jacopo Hu, George Mylonas, Daniel Elson
AI summary
Problem
Surgical robots suffer from inherent kinematic inaccuracies caused by unreliable internal sensors, which traditional calibration methods struggle to correct across varying clinical environments.
Approach
The authors train a low-level visuomotor controller in simulation using a teacher-student paradigm, where an RL teacher guides an imitation learning student to fuse imperfect proprioceptive data with reliable 2D visual keypoints for real-time error correction.
Key results
- A teacher-student visuomotor policy that compensates for kinematic errors using external visual feedback
- Successful sim-to-real deployment on a physical dVRK platform with competitive tracking performance
- Robust generalization to workspace relocation and varying camera viewpoints
- Validation on a modified peg transfer task demonstrating feasibility for surgical automation
Why it matters
This calibration-free control strategy provides a reliable foundation for advancing autonomous surgical robots and reducing surgeon workload in minimally invasive procedures.
Abstract
Robot-Assisted Surgery is integral to modern minimally invasive procedures, with automation emerging as the next frontier to enhance precision and reduce surgeon fatigue. This evolution is largely impeded by the inherent kinematic inaccuracies of surgical robots, where unreliable in- ternal sensors lead to significant control errors. While previous methods attempted to mitigate these issues through complex model-based calibration, they often suffer from high cost and limited effectiveness. This work utilises a learning-policy to actively compensate for hardware inaccuracies using closed- loop visual feedback that was trained from a teacher-student learning framework. The policy can fuse unreliable internal readings with precise external visual data, allowing it to correct for kinematic errors in real time without needing a perfect physical model. The learned policy was successfully deployed on the da Vinci Research Kit, where experiments validated the fundamental feasibility of using external vision to overcome internal sensor deficits. This research provides a foundational and reliable control methodology, paving the way for more advanced and robust surgical automation.