Self-supervised monocular depth estimation in gastroendoscopy using GAN-augmented images
Self-supervised monocular depth estimation in gastroendoscopy using GAN-augmented images
复制标题
DOI：
10.1117/12.2579317
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Aji Resindra Widya;Yusuke Monno;M. Okutomi;Sho Suzuki;T. Gotoda;Kenji Miki
中科院分区：
文献类型：
--
作者：
Aji Resindra Widya;Yusuke Monno;M. Okutomi;Sho Suzuki;T. Gotoda;Kenji Miki
胃镜检查是黄金标准程序，使医生能够调查病人的胃内。从内窥镜图像进行单目深度估计使得能够同时采集RGB和深度数据，这可以提高内窥镜用于各种潜在诊断应用的能力，例如RGB-D数据采集朝向用于病变定位的全胃3D重建和用于病变检查的局部视图扩展。因此，基于深度学习的方法正在获得牵引力，以在单眼内窥镜检查中提供深度信息。由于在临床环境中很难获得真实RGB和深度图像对，因此计算机生成（CG）数据通常用于训练深度估计网络。然而，CG数据在生成逼真的RGB和深度数据方面具有限制。在本文中，我们提出了一种新的数据生成策略，用于自我监督训练，以预测胃镜检查的深度。为了获得用于训练的密集参考深度数据，我们首先通过利用用靛蓝胭脂红（IC）蓝染料喷涂的彩色内窥镜图像来重建整个胃3D模型。然后，我们使用CycleGAN从彩色内窥镜图像生成虚拟无IC图像，以使我们的深度估计网络适用于没有IC染料的一般内窥镜图像。我们的实验表明，我们提出的方法实现了合理的深度预测的chromosendoscopic和一般的白光内窥镜图像。
Gastroendoscopy is the golden standard procedure that enables medical doctors to investigate the inside of a patient's stomach. Monocular depth estimation from an endoscopic image enables the simultaneous acquisition of RGB and depth data, which can boost the capability of the endoscopy for various potential diagnostic applications, such as the RGB-D data acquisition toward whole stomach 3D reconstruction for lesion localization and local view expansion for lesion inspection. Therefore, deep-learning-based approaches are gaining traction to provide depth information in monocular endoscopy. Since it is very difficult to obtain ground-truth RGB and depth image pairs in clinical settings, computer-generated (CG) data is usually used for training the depth estimation network. However, CG data has a limitation to generate realistic RGB and depth data. In this paper, we propose a novel data generation strategy for self-supervised training to predict the depth in gastroendoscopy. To obtain dense reference depth data for training, we first reconstruct a whole stomach 3D model by exploiting chromoendoscopic images sprayed with indigo carmine (IC) blue dye. We then generate virtual no-IC images from chromoendoscopic images using CycleGAN to make our depth estimation network applicable to general endoscopic images without IC dye. We experimentally demonstrate that our proposed approach achieves plausible depth prediction on both chromoendoscopic and general white-light endoscopic images.