Robotic Needle Insertion with 2D Ultrasound � 3D CT Fusion Guidance

Puncture robots pave a new way for stable, accurate and safe percutaneous liver tumor puncture operation. However, affected by respiratory motion, intraoperative accurate location of the tumor and its surrounding anatomical structures remains a difficult problem in existing robot-assisted puncture operations. In this paper, a dual-arm robotic needle insertion system with guidance of intraoperative 2D ultrasound (US) and preoperative 3D computed tomography (CT) fusion is proposed, addressing the shortcomings of existing puncture robots. To deal with the challenge of cross-modal and cross-dimensional registration between 2D US and 3D CT, a decoupled two-stage registration approach combining initial vessel structure-based 3D US – 3D CT registration with intraoperative intensity-based 2D US - 3D US registration is proposed. To achieve fast and robust ultrasound probe calibration, a method based on an improved N-wire phantom is proposed. Twenty puncture experiments are performed in different breath-holding positions on a respiratory motion simulation platform, and experimental results show that the mean puncture error is 2.48 mm, which can meet the requirements in a wide of clinical scenarios. This work was supported by the National Key Research and Development Program of China (No. 2022YFB4703500), National Natural Science Foundation of China (Nos. 62273328, U21A20489), Guangdong Basic and Applied Basic Research Foundation (Nos. 2021B1515130003, 2021B1515120011, 2021A1515012264), Shenzhen Science and Technology Program (Nos. JCYJ20200109115201707, JCYJ20220818101408019), Research Grants Council of the Hong Kong Special Administrative Region (No. T45-401/22-N), and Hong Kong ITF Fund (No. GHP/080/20SZ). This work is also supported by CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology. (Corresponding author: Baoliang Zhao and Ying Hu.) Long Lei is with Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China, he is also with Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong 999077, China (e-mail: long.lei@siat.ac.cn). Baoliang Zhao, Xiaozhi Qi, Peng Zhang, Qiong Wang and Ying Hu are with Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China (bl.zhao@siat.ac.cn; xz.qi@siat.ac.cn; zhangpeng@siat.ac.cn; wangqiong@siat.ac.cn; ying.hu@siat.ac.cn). Rui Mi and Hai Ye are with Department of Radiology, Shenzhen University General Hospital, Shenzhen University Clinical Medical Academy, Shenzhen 518055, China (e-mail: mirror@szu.edu.cn; yehai@szu.edu.cn). Pheng-Ann Heng is with Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong 999077, China, he is also with Guangdong-Hong Kong-Macao Joint Laboratory of Human Machine Intelligence-Synergy Systems, CUHK, Hong Kong 999077, China (e-mail: pheng@cse.cuhk.edu.hk). Note to Practitioners—In clinical percutaneous liver tumor puncture operation, due to the lack of real-time and clear image guidance, it is difficult to locate the tumor and its surrounding vital anatomical structures. In addition, the stability and accuracy of manual operation are poor. The development of a puncture robot is an effective solution for these problems. However, existing CT and magnetic resonance imaging (MRI) guided robots do not consider the tumor localization errors caused by inconsistent breath-holding positions between preoperative scan period and intraoperative puncture period, and US guided robots are limited by the poor image quality and the narrow field of vision. In this paper, a dual-arm robotic needle insertion system with guidance of intraoperative 2D US and preoperative 3D CT fusion is proposed. This system can take advantage of the real-time ultrasound and clear CT images at the same time, and can provide real-time, clear and all-round guidance for percutaneous liver tumor puncture operation, which has obvious advantages over the existing puncture robots. Phantom experiments have been completed and animal experiments will be carried out in the future.