Evaluation of Surgical Skills Using Machine Learning and Interpretation of Results with Explainable AI in Practical Laparoscopic Surgery Training

To facilitate efficient laparoscopic surgical educa- tion, a system was developed that utilizes machine learning to classify surgical skill levels—novice, intermediate, and ex- pert—based on the motion dynamics of surgical instruments. This system not only categorizes surgical proficiency but also incorporates SHAP, an Explainable AI technique, to provide insights into the rationale behind each classification result. For the machine learning dataset, the movements of four surgical instruments were recorded using a motion capture (mocap) system during total nephrectomy training sessions conducted on 46 cadaveric specimens prepared for laparoscopic surgery. The entire nephrectomy procedure was divided into three distinct processes: colon mobilization (Process 1), renal vascular dissection (Process 2), and incision and removal of the remaining tissues (Process 3). Surgical skill analysis was performed separately for each phase. Surgeons were categorized into three groups based on their prior experience with laparo- scopic procedures: novices (0–9 cases), intermediates (10–49 cases), and experts (50 or more cases). A total of 111 features were extracted from the instrument motion data for each phase, and comparative analyses were conducted across the three groups. Multiple machine learning approaches—including Support Vector Machine (SVM), Principal Component Analysis followed by SVM (PCA-SVM), and Random Forest—were employed to develop models for classifying surgeons into three distinct skill levels. The classification performance of these models was subsequently validated. The results revealed that features related to efficiency and speed significantly contributed to differences in surgical skill levels. The developed system enables quantitative comparison and visualization of specific instrument characteristics. This system contributes to intelligent integration of surgical education and Explainable AI, providing actionable feedback for skill improvement. Lingbo Yan, Koki Ebina, Taku Senoo, Atsushi Konno are with the Grad- uate School of Information Science and Technology, Hokkaido University, Sapporo, Japan. llingboyan@gmail.com, konno@ssi.ist.hokudai.ac.jp Takashige Abe, Masafumi Kon, Madoka Higuchi, Kiyohiko Hotta, Naoya Iwahara, Hiroshi Kikuchi, Haruka Miyata, Ryuji Matsumoto, Takahiro Osawa, Sachiyo Murai, Toshiaki Shichinohe, Soichi Murakami, Masahiko Watanabe are with the Graduate School of Medicine, Hokkaido University, Sapporo, Japan Jun Furumido is with Department of Urology, Asahikawa Kousei Hospi- tal, Asahikawa, Japan. Shunsuke Komizunai is with Faculty of Engineering and Design, Kagawa University, Takamatsu, Japan. Teppei Tsujita is with Department of Mechanical Engineering, National Defense Academy of Japan, Yokosuka, Japan. Kazuya Sase is with Faculty of Engineering, Tohoku Gakuin University, Sendai, Japan. Xiaoshuai Chen is with the Graduate School of Science and Technology, Hirosaki University, Hirosaki, Japan. Optical trinocular motion capture camera OptiTrack V120: Trio Surgeon Endoscope operator Surgical instruments with markers Cadaver Endoscope Monitor Fig. 1: Laparoscopic surgical training using cadaver.