D2-Net: Dual Disentanglement Network for Brain Tumor Segmentation With Missing Modalities
D2-Net: Dual Disentanglement Network for Brain Tumor Segmentation With Missing Modalities
复制标题
DOI：
10.1109/tmi.2022.3175478
复制
发表时间：
2022-10-01
影响因子：
10.6
通讯作者：
Yuan, Yixuan
中科院分区：
文献类型：
--
作者：
Yang, Qiushi;Guo, Xiaoqing;Yuan, Yixuan
多模态磁共振成像（MRI）可以为脑肿瘤的自动分割提供补充信息，这对诊断和预后至关重要。而丢失模态数据在临床实践中是常见的，并且它可能导致依赖于完整模态数据的大多数先前方法的崩溃。目前最先进的方法通过融合多模态图像和特征来学习肿瘤区域的共享表示来科普丢失模态的情况，这通常忽略了明确捕获模态和肿瘤区域之间的相关性。受模态信息在分割不同肿瘤区域中发挥不同作用的启发，我们的目标是明确利用各种模态特定信息和肿瘤特定知识之间的相关性进行分割。为此，我们提出了一个双解纠缠网络（D-2-Net）的脑肿瘤分割与丢失的方式，它包括一个模态解纠缠阶段（MD阶段）和肿瘤区域解纠缠阶段（TD阶段）。在MD-Stage中，设计了一种空间-频率联合模态对比学习方案，以直接从MRI数据中解耦模态特定信息。为了分解肿瘤特异性表示并提取有区别的整体特征，我们提出了一种在TD阶段中的亲和度引导的密集肿瘤区域知识蒸馏机制，通过将解纠缠的二元教师网络的特征与整体学生网络对齐。通过明确地发现模态和肿瘤区域之间的关系，即使某些模态丢失，我们的模型也可以学习足够的分割信息。在公共BraTS-2018数据库上进行的广泛实验证明了我们的框架在缺失模态情况下优于最先进的方法。代码可在https://github.com/CityU-AIM-Group/D2Net上获得。
Multi-modal Magnetic Resonance Imaging (MRI) can provide complementary information for automatic brain tumor segmentation, which is crucial for diagnosis and prognosis. While missing modality data is common in clinical practice and it can result in the collapse of most previous methods relying on complete modality data. Current state-of-the-art approaches cope with the situations of missing modalities by fusing multi-modal images and features to learn shared representations of tumor regions, which often ignore explicitly capturing the correlations among modalities and tumor regions. Inspired by the fact that modality information plays distinct roles to segment different tumor regions, we aim to explicitly exploit the correlations among various modality-specific information and tumor-specific knowledge for segmentation. To this end, we propose a Dual Disentanglement Network (D-2-Net) for brain tumor segmentation with missing modalities, which consists of a modality disentanglement stage (MD-Stage) and a tumor-region disentanglement stage (TD-Stage). In the MD-Stage, a spatial-frequency joint modality contrastive learning scheme is designed to directly decouple the modality-specific information from MRI data. To decompose tumor-specific representations and extract discriminative holistic features, we propose an affinity-guided dense tumor-region knowledge distillation mechanism in the TD-Stage through aligning the features of a disentangled binary teacher network with a holistic student network. By explicitly discovering relations among modalities and tumor regions, our model can learn sufficient information for segmentation even if some modalities are missing. Extensive experiments on the public BraTS-2018 database demonstrate the superiority of our framework over state-of-the-art methods in missing modalities situations. Codes are available at https://github.com/CityU-AIM-Group/D2Net.