Graph Convolutional Nets for Tool Presence Detection in Surgical Videos
Graph Convolutional Nets for Tool Presence Detection in Surgical Videos
复制标题
DOI：
10.1007/978-3-030-20351-1_36
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Sheng Wang;Zheng Xu;Chaochao Yan;Junzhou Huang
中科院分区：
文献类型：
--
作者：
Sheng Wang;Zheng Xu;Chaochao Yan;Junzhou Huang
手术工具存在检测是自动手术视频内容分析的关键问题之一。解决这个问题有利于许多应用，如手术器械使用的评估和自动手术报告生成。考虑到每个视频仅在帧级被稀疏地标记的事实，这意味着只有一小部分视频帧将被正确地标记，现有方法仅将该问题建模为图像（帧）分类问题，而不考虑手术视频中的时间信息。在本文中，我们提出了一个深度神经网络模型，利用手术视频的空间和时间信息进行手术工具存在检测。该模型利用图卷积网络（GCN）沿着时间维度，通过考虑连续视频帧之间的关系来学习更好的特征。据我们所知，这是第一个以视频作为输入来解决手术工具存在检测问题的工作。我们的实验表明，就业的时间信息提供了一个显着的改善这个问题，所提出的方法实现了更好的性能比所有国家的最先进的方法。
Surgical tool presence detection is one of the key problems in automatic surgical video content analysis. Solving this problem benefits many applications such as the evaluation of surgical instrument usage and automatic surgical report generation. Given the fact that each video is only sparsely labeled at the frame level, meaning that only a small portion of video frames will be properly labeled, existing approaches only model this problem as an image (frame) classification problem without considering temporal information in surgical videos. In this paper, we propose a deep neural network model utilizing both spatial and temporal information from surgical videos for surgical tool presence detection. The proposed model uses Graph Convolutional Networks (GCNs) along the temporal dimension to learn better features by considering the relationship between continuous video frames. To the best of our knowledge, this is the first work taking videos as input to solve the surgical tool presence detection problem. Our experiments demonstrate the employment of temporal information offers a significant improvement to this problem, and the proposed approach achieves better performance than all state-of-the-art methods.