Enabling machine learning in X-ray-based procedures via realistic simulation of image formation
Enabling machine learning in X-ray-based procedures via realistic simulation of image formation
复制标题
DOI：
10.1007/s11548-019-02011-2
复制
发表时间：
2019-09-01
影响因子：
3
通讯作者：
Navab, Nassir
中科院分区：
文献类型：
--
作者：
Unberath, Mathias;Zaech, Jan-Nico;Navab, Nassir
基于机器学习的方法现在在与诊断放射学相关的大多数学科中优于竞争方法。然而，图像引导程序尚未从深度学习的出现中获益，特别是因为用于程序引导的图像没有存档，因此无法用于学习，即使它们可用，由于大量的数据，注释也将是一个严峻的挑战。来自3D CT的X射线图像的计算机模拟是使用真实临床X射线照片的一种有趣的替代方法，因为标记非常容易并且可能随时可用。方法我们扩展了我们的框架，用于从高分辨率CT（称为DeepDRR）中快速逼真地模拟透视，并具有工具建模功能。该框架是公开可用的，开源的，并与深度学习原生的软件平台紧密集成，即，Python、PyTorch和PyCuda。DeepDRR分别依赖机器学习来进行3D和2D的材料分解和散射估计，但使用分析前向投影和噪声注入来确保可接受的计算时间。在两个X射线图像分析任务中，即（1）解剖标志检测和（2）机器人末端执行器的分割和定位，我们证明了在DeepDRR上训练的卷积神经网络（ConvNets）可以很好地推广到真实的数据，而无需重新训练或域自适应。为此，我们使用完全相同的训练协议在naive和DeepDRR上训练ConvNets，并比较它们在使用临床C形臂X射线系统获取的尸体标本数据上的性能。结果我们的研究结果是一致的两个考虑的任务。当在相应的合成测试集上进行评价时，所有ConvNet的性能均相似。然而，当应用于尸体解剖结构的真实的X射线照片时，在DeepDRR上训练的ConvNets显著优于在naive DRR上训练的ConvNets（p
Purpose Machine learning-based approaches now outperform competing methods in most disciplines relevant to diagnostic radiology. Image-guided procedures, however, have not yet benefited substantially from the advent of deep learning, in particular because images for procedural guidance are not archived and thus unavailable for learning, and even if they were available, annotations would be a severe challenge due to the vast amounts of data. In silico simulation of X-ray images from 3D CT is an interesting alternative to using true clinical radiographs since labeling is comparably easy and potentially readily available. Methods We extend our framework for fast and realistic simulation of fluoroscopy from high-resolution CT, called DeepDRR, with tool modeling capabilities. The framework is publicly available, open source, and tightly integrated with the software platforms native to deep learning, i.e., Python, PyTorch, and PyCuda. DeepDRR relies on machine learning for material decomposition and scatter estimation in 3D and 2D, respectively, but uses analytic forward projection and noise injection to ensure acceptable computation times. On two X-ray image analysis tasks, namely (1) anatomical landmark detection and (2) segmentation and localization of robot end-effectors, we demonstrate that convolutional neural networks (ConvNets) trained on DeepDRRs generalize well to real data without re-training or domain adaptation. To this end, we use the exact same training protocol to train ConvNets on naive and DeepDRRs and compare their performance on data of cadaveric specimens acquired using a clinical C-arm X-ray system. Results Our findings are consistent across both considered tasks. All ConvNets performed similarly well when evaluated on the respective synthetic testing set. However, when applied to real radiographs of cadaveric anatomy, ConvNets trained on DeepDRRs significantly outperformed ConvNets trained on naive DRRs (p