Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery.
Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery.
复制标题
DOI：
10.1007/s11548-016-1393-4
复制
发表时间：
2016-06
影响因子：
3
通讯作者：
Stoyanov D
中科院分区：
文献类型：
--
作者：
Du X;Allan M;Dore A;Ourselin S;Hawkes D;Kelly JD;Stoyanov D
用于增强型微创手术（MIS）的计算机辅助干预需要跟踪手术器械。仪器跟踪是一个具有挑战性的问题，在传统的和机器人辅助的MIS，但基于视觉的方法是一个有前途的解决方案，最小的硬件集成要求。然而，基于视觉的方法遭受漂移，并且在遮挡、阴影和快速运动的情况下，它们可能会遭受完全的跟踪失败。在本文中，我们开发了一个二维跟踪器的基础上，广义Hough变换使用SIFT功能，既可以处理复杂的环境变化，并从跟踪失败恢复。我们使用它来初始化每帧的3D跟踪器，这使我们能够在长序列中甚至在遮挡期间恢复3D仪器姿势。我们定量验证我们的方法在2D和3D与从DVRK控制器收集的体外数据，以及提供定性验证机器人辅助的体内数据。我们证明，从我们的扩展序列，我们的方法提供了无漂移的鲁棒性和准确的跟踪。我们基于遮挡的序列还证明了我们的方法可以从基于遮挡的失败中恢复。在这两种情况下，我们都显示了单独使用3D跟踪的改进，这表明将2D和3D跟踪相结合是应对手术器械跟踪挑战的有前途的解决方案。本文的在线版本（doi：10.1007/s11548-016-1393-4）包含补充材料，可供授权用户使用。
Computer-assisted interventions for enhanced minimally invasive surgery (MIS) require tracking of the surgical instruments. Instrument tracking is a challenging problem in both conventional and robotic-assisted MIS, but vision-based approaches are a promising solution with minimal hardware integration requirements. However, vision-based methods suffer from drift, and in the case of occlusions, shadows and fast motion, they can be subject to complete tracking failure. In this paper, we develop a 2D tracker based on a Generalized Hough Transform using SIFT features which can both handle complex environmental changes and recover from tracking failure. We use this to initialize a 3D tracker at each frame which enables us to recover 3D instrument pose over long sequences and even during occlusions. We quantitatively validate our method in 2D and 3D with ex vivo data collected from a DVRK controller as well as providing qualitative validation on robotic-assisted in vivo data. We demonstrate from our extended sequences that our method provides drift-free robust and accurate tracking. Our occlusion-based sequences additionally demonstrate that our method can recover from occlusion-based failure. In both cases, we show an improvement over using 3D tracking alone suggesting that combining 2D and 3D tracking is a promising solution to challenges in surgical instrument tracking. The online version of this article (doi:10.1007/s11548-016-1393-4) contains supplementary material, which is available to authorized users.