Vitreoretinal Surgical Instrument Tracking in Three Dimensions Using Deep Learning.
Vitreoretinal Surgical Instrument Tracking in Three Dimensions Using Deep Learning.
复制标题
DOI：
10.1167/tvst.12.1.20
复制
发表时间：
2023-01-03
影响因子：
3
通讯作者：
Browne, Andrew W.
中科院分区：
文献类型：
--
作者：
Baldi, Pierre F.;Abdelkarim, Sherif;Liu, Junze;To, Josiah K.;Ibarra, Marialejandra Diaz;Browne, Andrew W.
评估基于人工智能的视频分析的潜力，以确定手术器械在三维玻璃体空间中移动时的特征。我们设计并制造了一个模型眼，在这个模型眼中，我们记录了许多手术器械在整个眼睛中移动的精心设计的视频。我们标记了视频的每一帧来描述手术工具的特征：工具类型、位置、深度和插入侧边。我们训练了两种不同的深度学习模型来预测每种工具的特征，并在图像子集上评估模型的性能。在训练集上，分类模型在x-y区域的准确率为84%，在深度区域的准确率为97%，在仪器类型区域的准确率为100%，在插入侧度区域的准确率为100%。验证数据集上的分类模型在x-y区域的准确率为83%，深度的准确率为96%，仪器类型的准确率为100%，插入侧向度的准确率为100%。近距离检测模型以67帧/秒的速度运行，大多数仪器的精度高于75%，平均精度达到79.3%。我们证明，经过训练的模型可以跟踪手术器械在三维空间中的运动，并确定器械深度、尖端位置、器械插入侧度和器械类型。模型的性能几乎是即时的，证明了进一步研究应用于现实世界的手术视频。深度学习为手术过程中基于软件的安全反馈机制或提取手术技术指标的能力提供了潜力，这些指标可以指导研究以优化手术结果。
To evaluate the potential for artificial intelligence-based video analysis to determine surgical instrument characteristics when moving in the three-dimensional vitreous space. We designed and manufactured a model eye in which we recorded choreographed videos of many surgical instruments moving throughout the eye. We labeled each frame of the videos to describe the surgical tool characteristics: tool type, location, depth, and insertional laterality. We trained two different deep learning models to predict each of the tool characteristics and evaluated model performances on a subset of images. The accuracy of the classification model on the training set is 84% for the x–y region, 97% for depth, 100% for instrument type, and 100% for laterality of insertion. The accuracy of the classification model on the validation dataset is 83% for the x–y region, 96% for depth, 100% for instrument type, and 100% for laterality of insertion. The close-up detection model performs at 67 frames per second, with precision for most instruments higher than 75%, achieving a mean average precision of 79.3%. We demonstrated that trained models can track surgical instrument movement in three-dimensional space and determine instrument depth, tip location, instrument insertional laterality, and instrument type. Model performance is nearly instantaneous and justifies further investigation into application to real-world surgical videos. Deep learning offers the potential for software-based safety feedback mechanisms during surgery or the ability to extract metrics of surgical technique that can direct research to optimize surgical outcomes.
登录
查看更多内容
影响因子：
82.9
作者：
Russell, Stephen R.;Drack, Arlene, V;Cideciyan, Artur, V;Jacobson, Samuel G.;Leroy, Bart P.;Van Cauwenbergh, Caroline;Ho, Allen C.;Dumitrescu, Alina, V;Han, Ian C.;Martin, Mitchell;Pfeifer, Wanda L.;Sohn, Elliott H.;Walshire, Jean;Garafalo, Alexandra, V;Krishnan, Arun K.;Powers, Christian A.;Sumaroka, Alexander;Roman, Alejandro J.;Vanhonsebrouck, Eva;Jones, Eltanara;Nerinckx, Fanny;De Zaeytijd, Julie;Collin, Rob W. J.;Hoyng, Carel;Adamson, Peter;Cheetham, Michael E.;Schwartz, Michael R.;den Hollander, Wilhelmina;Asmus, Friedrich;Platenburg, Gerard;Rodman, David;Girach, Aniz
通讯作者：
Girach, Aniz
DOI：
10.2147/opth.s292527
发表时间：
2021
期刊：
Clinical ophthalmology (Auckland, N.Z.)
影响因子：
--
作者：
Aleman TS;Miller AJ;Maguire KH;Aleman EM;Serrano LW;O'Connor KB;Bedoukian EC;Leroy BP;Maguire AM;Bennett J
通讯作者：
Bennett J
影响因子：
1.6
作者：
Apfelbaum, Henry;Pelah, Adar;Peli, Eli
通讯作者：
Peli, Eli
影响因子：
1.8
作者：
Peli E;Apfelbaum H;Berson EL;Goldstein RB
通讯作者：
Goldstein RB
影响因子：
4.2
作者：
Chung, Daniel C.;Bertelsen, Mette;Reape, Kathleen Z.
通讯作者：
Reape, Kathleen Z.