Markerless Suture Needle Tracking From A Robotic Endoscope Based On Deep Learning
Markerless Suture Needle Tracking From A Robotic Endoscope Based On Deep Learning
复制标题
DOI：
10.1109/ismr57123.2023.10130199
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Yiwei Jiang;Haoying Zhou;G. Fischer
中科院分区：
文献类型：
--
作者：
Yiwei Jiang;Haoying Zhou;G. Fischer
自二十年前以来，机器人辅助手术的进步一直在迅速增长。最近，机器人手术任务的自动化已经成为研究的焦点。在这一领域，手术工具的检测和跟踪对于自主系统规划和执行手术至关重要。例如，知道针的位置和姿态是自动识别系统抓住它并执行识别任务的先决条件。在本文中，我们提出了一种基于深度学习和点对点配准的新方法，用于跟踪机器人内窥镜（da芬奇机器人手术系统的内窥镜摄像机操纵器）中金属缝合针的6个自由度（DOF）姿态，而无需任何标记的帮助。所提出的方法在2021-2022年WARNET外科机器人挑战赛提供的标准模拟手术环境中实施和评估，从而证明了将其转化为现实世界场景的潜力。构建了一个包含836幅从模拟场景中采集的图像的自定义数据集，该数据集包含姿势和关键点信息的地面真实值，用于训练神经网络模型。最好的管道实现了1.76 mm的平均位置误差，而平均方向误差为8.55度，它可以在PC上运行高达10 Hz。
Advancements in robot-assisted surgery have been rapidly growing since two decades ago. More recently, the automation of robotic surgical tasks has become the focus of research. In this area, the detection and tracking of a surgical tool are crucial for an autonomous system to plan and perform a procedure. For example, knowing the position and posture of a needle is a prerequisite for an automatic suturing system to grasp it and perform suturing tasks. In this paper, we proposed a novel method, based on Deep Learning and Point-to-point Registration, to track the 6 degrees of freedom (DOF) pose of a metal suture needle from a robotic endoscope (an Endoscopic Camera Manipulator from the da Vinci Robotic Surgical Systems), without the help of any marker. The proposed approach was implemented and evaluated in a standard simulated surgical environment provided by the 2021–2022 AccelNet Surgical Robotics Challenge, thus demonstrates the potential to be translated into a real-world scenario. A customized dataset containing 836 images collected from the simulated scene with ground truth of poses and key points information was constructed to train the neural network model. The best pipeline achieved an average position error of 1.76 mm while the average orientation error is 8.55 degrees, and it can run up to 10 Hz on a PC.