Co-Learning Feature Fusion Maps From PET-CT Images of Lung Cancer
Co-Learning Feature Fusion Maps From PET-CT Images of Lung Cancer
复制标题
DOI：
10.1109/tmi.2019.2923601
复制
发表时间：
2020-01-01
影响因子：
10.6
通讯作者：
Kim, Jinman
中科院分区：
文献类型：
--
作者：
Kumar, Ashnil;Fulham, Michael;Kim, Jinman
用于计算机辅助诊断应用的多模态正电子发射断层扫描和计算机断层扫描（PET-CT）图像的分析（例如，检测和分割）需要将PET检测异常区域的灵敏度与来自CT的解剖定位相结合。用于PET-CT图像分析的当前方法或者单独处理模态，或者基于关于图像分析任务的知识融合来自每个模态的信息。这些方法通常不考虑跨不同模态编码不同信息的空间变化视觉特性，这些模态在不同位置处具有不同优先级。例如，肺中的高异常PET摄取比心脏中的生理PET摄取对于肿瘤检测更有意义。我们的目标是通过一种新的监督卷积神经网络（CNN）来改善多模态PET-CT中互补信息的融合，该网络学习融合多模态医学图像分析的互补信息。我们的CNN首先对特定于模态的特征进行编码，然后使用它们来导出一个空间变化的融合图，该融合图量化了不同空间位置上每种模态特征的相对重要性。然后将这些融合图与特定于模态的特征图相乘，以获得不同位置处的互补多模态信息的表示，然后可以将其用于图像分析。我们使用肺癌PET-CT图像数据集评估了CNN检测和分割具有不同融合要求的多个区域（肺，纵隔和肿瘤）的能力。我们将我们的方法与多模态图像融合（融合输入（FS），多分支（MB）技术和多通道（MC）技术）和分割的基线技术进行了比较。我们的研究结果表明，我们的CNN具有比融合基线（FS：99.00%，MB：99.08%和TC：98.92%）显著更高的前景检测准确性（99.29%，p < 0.05），并且比最近的PET-CT肿瘤分割方法具有显著更高的Dice评分（63.85%）。
The analysis of multi-modality positron emission tomography and computed tomography (PET-CT) images for computer-aided diagnosis applications (e.g., detection and segmentation) requires combining the sensitivity of PET to detect abnormal regions with anatomical localization from CT. Current methods for PET-CT image analysis either process the modalities separately or fuse information from each modality based on knowledge about the image analysis task. These methods generally do not consider the spatially varying visual characteristics that encode different information across different modalities, which have different priorities at different locations. For example, a high abnormal PET uptake in the lungs is more meaningful for tumor detection than physiological PET uptake in the heart. Our aim is to improve the fusion of the complementary information in multi-modality PET-CT with a new supervised convolutional neural network (CNN) that learns to fuse complementary information for multi-modality medical image analysis. Our CNN first encodes modality-specific features and then uses them to derive a spatially varying fusion map that quantifies the relative importance of each modality's feature across different spatial locations. These fusion maps are then multiplied with the modality-specific feature maps to obtain a representation of the complementary multi-modality information at different locations, which can then be used for image analysis. We evaluated the ability of our CNN to detect and segment multiple regions (lungs, mediastinum, and tumors) with different fusion requirements using a dataset of PET-CT images of lung cancer. We compared our method to baseline techniques for multi-modality image fusion (fused inputs (FSs), multi-branch (MB) techniques, and multi-channel (MC) techniques) and segmentation. Our findings show that our CNN had a significantly higher foreground detection accuracy (99.29%, p < 0.05) than the fusion baselines (FS: 99.00%, MB: 99.08%, and TC: 98.92%) and a significantly higher Dice score (63.85%) than the recent PET-CT tumor segmentation methods.