SLAM Endoscopy enhanced by adversarial depth prediction
SLAM Endoscopy enhanced by adversarial depth prediction
复制标题
通过对抗性深度预测增强 SLAM 内窥镜检查
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
N. Durr
中科院分区：
文献类型：
--
作者：
Richard J. Chen;Taylor L. Bobrow;Thomas L. Athey;Faisal Mahmood;N. Durr
医学内窥镜仍然是一个具有挑战性的应用，同时定位和映射（SLAM）由于图像特征的稀疏性和大小的限制，防止直接的深度传感。我们提出了一种SLAM方法，该方法结合了由应用于单眼内窥镜图像的逆向训练的卷积神经网络（CNN）进行的深度预测。深度网络使用简单结肠模型的合成图像进行训练，然后使用从人类结肠的计算机断层扫描测量结果渲染的域随机化、照片级真实感图像进行微调。每个图像都与无错误的深度图配对，用于监督对抗学习。然后将单目RGB图像与相应的深度预测融合，从而在内窥镜通过胃肠道前进时实现密集重建和拼接。我们的初步结果表明，将单目深度估计纳入SLAM架构可以实现内窥镜场景的密集重建。
Medical endoscopy remains a challenging application for simultaneous localization and mapping (SLAM) due to the sparsity of image features and size constraints that prevent direct depth-sensing. We present a SLAM approach that incorporates depth predictions made by an adversarially-trained convolutional neural network (CNN) applied to monocular endoscopy images. The depth network is trained with synthetic images of a simple colon model, and then fine-tuned with domain-randomized, photorealistic images rendered from computed tomography measurements of human colons. Each image is paired with an error-free depth map for supervised adversarial learning. Monocular RGB images are then fused with corresponding depth predictions, enabling dense reconstruction and mosaicing as an endoscope is advanced through the gastrointestinal tract. Our preliminary results demonstrate that incorporating monocular depth estimation into a SLAM architecture can enable dense reconstruction of endoscopic scenes.