Depth estimation from single-shot monocular endoscope image using image domain adaptation and edge-aware depth estimation
Depth estimation from single-shot monocular endoscope image using image domain adaptation and edge-aware depth estimation
复制标题
DOI：
10.1080/21681163.2021.2012835
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
M. Oda;H. Itoh;Kiyohito Tanaka;H. Takabatake;M. Mori;H. Natori;K. Mori
中科院分区：
文献类型：
--
作者：
M. Oda;H. Itoh;Kiyohito Tanaka;H. Takabatake;M. Mori;H. Natori;K. Mori
摘要：我们提出了一种单镜头单目内窥镜图像的深度估计方法，该方法使用通过域适应的朗伯表面平移和使用多尺度边缘损失的深度估计。我们采用两步估计过程，包括来自未配对数据的朗伯表面平移和深度估计。器官表面的纹理和镜面反射会降低深度估计的准确性。我们将朗伯表面平移应用于内窥镜图像以去除这些纹理和反射。然后，我们使用全卷积网络（FCN）来估计深度。在 FCN 的训练过程中，提高估计图像和地面真实深度图像之间的对象边缘相似度对于获得更好的结果非常重要。我们引入了多尺度边缘损失函数来提高深度估计的准确性。我们使用真实的结肠镜图像定量评估了所提出的方法。估计的深度值与真实深度值成比例。此外，我们使用卷积神经网络将估计的深度图像应用于结肠镜图像的自动解剖位置识别。通过使用估计的深度图像，网络的识别准确率从 69.2% 提高到 74.1%。
ABSTRACT We propose a depth estimation method from a single-shot monocular endoscopic image using Lambertian surface translation by domain adaptation and depth estimation using multi-scale edge loss. We employ a two-step estimation process including Lambertian surface translation from unpaired data and depth estimation. The texture and specular reflection on the surface of an organ reduce the accuracy of depth estimations. We apply Lambertian surface translation to an endoscopic image to remove these texture and reflections. Then, we estimate the depth by using a fully convolutional network (FCN). During the training of the FCN, improvement of the object edge similarity between an estimated image and a ground truth depth image is important for getting better results. We introduced a muti-scale edge loss function to improve the accuracy of depth estimation. We quantitatively evaluated the proposed method using real colonoscopic images. The estimated depth values were proportional to the real depth values. Furthermore, we applied the estimated depth images to automated anatomical location identification of colonoscopic images using a convolutional neural network. The identification accuracy of the network improved from 69.2% to 74.1% by using the estimated depth images.