Deep Learning-based Image Segmentation on Multimodal Medical Imaging.
Deep Learning-based Image Segmentation on Multimodal Medical Imaging.
复制标题
DOI：
10.1109/trpms.2018.2890359
复制
发表时间：
2019-03
影响因子：
4.4
通讯作者：
Li Q
中科院分区：
文献类型：
--
作者：
Guo Z;Li X;Huang H;Guo N;Li Q
多模式医学成像技术在临床实践和研究中的应用越来越多。相应的多模式图像分析和集成学习方案得到了快速的发展，并为医学应用带来了独特的价值。基于最近深度学习方法在医学图像处理中的成功应用，我们首先在特征学习层、分类器层和决策层提出了一种具有跨模式融合的有监督多模式图像分析的算法框架。然后，我们设计并实现了一个基于深卷积神经网络(CNN)的图像分割系统，利用来自磁共振成像(MRI)、计算机断层扫描(CT)和正电子发射断层扫描(PET)的多模式图像对软组织肉瘤的病变进行轮廓划分。与用单模式图像训练的网络相比，用多模式图像训练的网络表现出更好的性能。对于肿瘤分割任务，在网络内进行图像融合(即在卷积或完全连通的层进行融合)通常比在网络输出(即投票)进行图像融合要好。本研究为多通道图像分析的设计和应用提供了实证指导。
Multi-modality medical imaging techniques have been increasingly applied in clinical practice and research studies. Corresponding multi-modal image analysis and ensemble learning schemes have seen rapid growth and bring unique value to medical applications. Motivated by the recent success of applying deep learning methods to medical image processing, we first propose an algorithmic architecture for supervised multi-modal image analysis with cross-modality fusion at the feature learning level, classifier level, and decision-making level. We then design and implement an image segmentation system based on deep Convolutional Neural Networks (CNN) to contour the lesions of soft tissue sarcomas using multi-modal images, including those from Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and Positron Emission Tomography (PET). The network trained with multi-modal images shows superior performance compared to networks trained with single-modal images. For the task of tumor segmentation, performing image fusion within the network (i.e. fusing at convolutional or fully connected layers) is generally better than fusing images at the network output (i.e. voting). This study provides empirical guidance for the design and application of multi-modal image analysis.