MouseGAN++: Unsupervised Disentanglement and Contrastive Representation for Multiple MRI Modalities Synthesis and Structural Segmentation of Mouse Brain
MouseGAN++: Unsupervised Disentanglement and Contrastive Representation for Multiple MRI Modalities Synthesis and Structural Segmentation of Mouse Brain
复制标题
MouseGAN：小鼠大脑多种 MRI 模式合成和结构分割的无监督解缠结和对比表示
DOI：
10.1109/tmi.2022.3225528
复制
发表时间：
2022-11
影响因子：
10.6
通讯作者：
Xiao-Yong Zhang
中科院分区：
文献类型：
--
作者：
Ziqi Yu;Xiaoyang Han;Shengjie Zhang;Jianfeng Feng;Tingying Peng;Xiao-Yong Zhang
在磁共振（MR）图像上分割小鼠大脑的精细结构对于描绘形态学区域，分析大脑功能以及理解它们之间的关系至关重要。与单一MRI模态相比，多模态MRI数据提供了互补的组织特征，可以被深度学习模型利用，从而产生更好的分割结果。然而，由于缺乏多模态小鼠脑MRI数据，使得小鼠脑精细结构的自动分割成为一项非常具有挑战性的任务。为了解决这个问题，有必要融合多模态MRI数据，以产生不同脑结构的区分对比。因此，我们提出了一种新的去纠缠和对比的基于gan的框架，命名为MouseGAN++，以保持结构的方式从单个MR模态合成多个MR模态，从而通过输入缺失模态和多模态融合提高分割性能。我们的结果表明，我们的方法的翻译性能优于最先进的方法。利用随后学习到的模态不变信息以及模态转换图像，MouseGAN++可以分割精细的大脑结构，平均dice系数分别为90.0% （T2w）和87.9% (T1w)，与目前最先进的算法相比，性能提高了约10%。我们的研究结果表明，MouseGAN++作为一种同时进行图像合成和分割的方法，可以以不配对的方式融合跨模态信息，并在没有多模态数据的情况下产生更强的鲁棒性。我们将我们的方法作为一个老鼠大脑结构分割工具发布在https://github.com/yu02019上供免费学术使用。
Segmenting the fine structure of the mouse brain on magnetic resonance (MR) images is critical for delineating morphological regions, analyzing brain function, and understanding their relationships. Compared to a single MRI modality, multimodal MRI data provide complementary tissue features that can be exploited by deep learning models, resulting in better segmentation results. However, multimodal mouse brain MRI data is often lacking, making automatic segmentation of mouse brain fine structure a very challenging task. To address this issue, it is necessary to fuse multimodal MRI data to produce distinguished contrasts in different brain structures. Hence, we propose a novel disentangled and contrastive GAN-based framework, named MouseGAN++, to synthesize multiple MR modalities from single ones in a structure-preserving manner, thus improving the segmentation performance by imputing missing modalities and multi-modality fusion. Our results demonstrate that the translation performance of our method outperforms the state-of-the-art methods. Using the subsequently learned modality-invariant information as well as the modality-translated images, MouseGAN++ can segment fine brain structures with averaged dice coefficients of 90.0% (T2w) and 87.9% (T1w), respectively, achieving around +10% performance improvement compared to the state-of-the-art algorithms. Our results demonstrate that MouseGAN++, as a simultaneous image synthesis and segmentation method, can be used to fuse cross-modality information in an unpaired manner and yield more robust performance in the absence of multimodal data. We release our method as a mouse brain structural segmentation tool for free academic usage at https://github.com/yu02019.
登录
查看更多内容
AMAP是用于注册和分割高分辨率小鼠脑数据的验证管道。
影响因子：
16.6
作者：
Niedworok CJ;Brown AP;Jorge Cardoso M;Osten P;Ourselin S;Modat M;Margrie TW
通讯作者：
Margrie TW
影响因子：
10.6
作者：
Sheikh, HR;Bovik, AC
通讯作者：
Bovik, AC
DOI：
--
发表时间：
2017-03
期刊：
--
影响因子：
--
作者：
Ming-Yu Liu;T. Breuel;J. Kautz
通讯作者：
Ming-Yu Liu;T. Breuel;J. Kautz
影响因子：
6
作者：
L. Maaten;Geoffrey E. Hinton
通讯作者：
L. Maaten;Geoffrey E. Hinton
DOI：
10.1007/978-3-030-87193-2_42
发表时间：
2021
期刊：
--
影响因子：
--
作者：
Ziqi Yu;Yuting Zhai;Xiaoyang Han;Tingying Peng;Xiao-Yong Zhang
通讯作者：
Ziqi Yu;Yuting Zhai;Xiaoyang Han;Tingying Peng;Xiao-Yong Zhang