Surgical Instrument Segmentation for Endoscopic Vision with Data Fusion of rediction and Kinematic Pose
Surgical Instrument Segmentation for Endoscopic Vision with Data Fusion of rediction and Kinematic Pose
复制标题
DOI：
10.1109/icra.2019.8794122
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Fangbo Qin;Yangming Li;Yun-Hsuan Su;De Xu;B. Hannaford
中科院分区：
文献类型：
--
作者：
Fangbo Qin;Yangming Li;Yun-Hsuan Su;De Xu;B. Hannaford
实时且鲁棒的手术器械分割是内窥镜视觉的一个重要问题。我们提出了一种融合卷积神经网络（CNN）预测和运动学姿态信息的仪器分割方法。首先，设计了 CNN 模型 ToolNet-C，它将在大量未标记图像上训练的卷积特征提取器和在少量标记图像上训练的像素分割器级联。其次，根据测量的运动学姿态实现器械主体到内窥镜图像上的轮廓投影。第三，提出了具有形状匹配似然和权重抑制的粒子滤波器用于数据融合，其估计细化了运动学姿态。精细化的姿势决定了准确的轮廓掩模，这是最终的分割输出。实验是使用手术导航系统、几种动物组织背景和清创器仪器进行的。
The real-time and robust surgical instrument segmentation is an important issue for endoscopic vision. We propose an instrument segmentation method fusing the convolutional neural networks (CNN) prediction and the kinematic pose information. First, the CNN model ToolNet-C is designed, which cascades a convolutional feature extractor trained over numerous unlabeled images and a pixel-wise segmentor trained on few labeled images. Second, the silhouette projection of the instrument body onto the endoscopic image is implemented based on the measured kinematic pose. Third, the particle filter with the shape matching likelihood and the weight suppression is proposed for data fusion, whose estimate refines the kinematic pose. The refined pose determines an accurate silhouette mask, which is the final segmentation output. The experiments are conducted with a surgical navigation system, several ani-mal-tissue backgrounds, and a debrider instrument.