Recurrent feature fusion learning for multi-modality pet-ct tumor segmentation
Recurrent feature fusion learning for multi-modality pet-ct tumor segmentation
复制标题
DOI：
10.1016/j.cmpb.2021.106043
复制
发表时间：
2021-03-19
影响因子：
6.1
通讯作者：
Kim, Jinman
中科院分区：
文献类型：
--
作者：
Bi, Lei;Fulham, Michael;Kim, Jinman
背景和目的：[18F]-氟脱氧葡萄糖（FDG）正电子发射断层摄影-计算机断层摄影（PET-CT）现在是对许多癌症进行分期的优选成像模式。Pet图像表征了肿瘤的葡萄糖代谢，而ct则描绘了肿瘤的互补解剖定位。肿瘤自动分割是计算机辅助诊断系统中图像分析的重要环节。最近，全卷积网络（fcns），其利用注释数据集和提取图像特征表示的能力，已成为肿瘤分割的最新技术。存在有限的支持多模态图像的基于FCN的方法，并且当前的方法主要集中于在各个阶段融合多模态图像特征，即，在FCN之前融合多模态图像特征的早期融合、具有融合的结果特征的后期融合以及在多个图像特征尺度上融合多模态图像特征的超融合。然而，早期和晚期融合方法具有固有的、有限的自由度来融合互补的多模态图像特征。超融合方法在不同的图像特征尺度上学习不同的图像特征，这可能导致不准确的分割，特别是在肿瘤具有异质纹理的情况下。研究方法：我们提出了一种递归融合网络（RFN），它由多个递归融合阶段组成，以渐进地融合互补的多模态图像特征和在各个递归融合阶段得到的中间分割结果：（1）递归融合阶段迭代地学习图像特征，然后改进后续的分割结果;以及（2）中间分割结果允许我们的方法专注于学习这些中间分割结果周围的多模态图像特征，这最小化了不一致特征学习的风险。结果：我们在两个经病理证实的非小细胞肺癌petct数据集上评估了我们的方法。我们将我们的方法与常用的融合方法（早期融合，晚期融合和超融合）以及各种网络骨干（resnet，densenet和3d-unet）上最先进的pet-ct肿瘤分割方法进行了比较。我们的研究结果表明，rfn提供了更准确的分割相比，现有的方法，并推广到不同的数据集。结论：我们表明，通过多个反复的融合阶段的学习允许反复重复使用多模态图像特征，从而细化肿瘤分割结果。我们还确定，我们的rfn产生一致的分割结果在不同的网络架构。(c)2021爱思唯尔有限公司版权所有。
Background and objective: [18f]-fluorodeoxyglucose (fdg) positron emission tomography-computed tomography (pet-ct) is now the preferred imaging modality for staging many cancers. Pet images characterize tumoral glucose metabolism while ct depicts the complementary anatomical localization of the tumor. Automatic tumor segmentation is an important step in image analysis in computer aided diagnosis systems. Recently, fully convolutional networks (fcns), with their ability to leverage annotated datasets and extract image feature representations, have become the state-of-the-art in tumor segmentation. There are limited fcn based methods that support multi-modality images and current methods have primarily focused on the fusion of multi-modality image features at various stages, i.e., early-fusion where the multi-modality image features are fused prior to fcn, late-fusion with the resultant features fused and hyper-fusion where multi-modality image features are fused across multiple image feature scales. Early and late-fusion methods, however, have inherent, limited freedom to fuse complementary multi-modality image features. The hyper-fusion methods learn different image features across different image feature scales that can result in inaccurate segmentations, in particular, in situations where the tumors have heterogeneous textures. Methods: we propose a recurrent fusion network (rfn), which consists of multiple recurrent fusion phases to progressively fuse the complementary multi-modality image features with intermediary segmentation results derived at individual recurrent fusion phases: (1) the recurrent fusion phases iteratively learn the image features and then refine the subsequent segmentation results; and, (2) the intermediary segmentation results allows our method to focus on learning the multi-modality image features around these intermediary segmentation results, which minimize the risk of inconsistent feature learning. Results: we evaluated our method on two pathologically proven non-small cell lung cancer petct datasets. We compared our method to the commonly used fusion methods (early-fusion, late-fusion and hyper-fusion) and the state-of-the-art pet-ct tumor segmentation methods on various network backbones (resnet, densenet and 3d-unet). Our results show that the rfn provides more accurate segmentation compared to the existing methods and is generalizable to different datasets. Conclusions: we show that learning through multiple recurrent fusion phases allows the iterative re-use of multi-modality image features that refines tumor segmentation results. We also identify that our rfn produces consistent segmentation results across different network architectures.(c) 2021 Elsevier B.V. All rights reserved.