Multimodal image synthesis based on disentanglement representations of anatomical and modality specific features, learned using uncooperative relativistic GAN.
Multimodal image synthesis based on disentanglement representations of anatomical and modality specific features, learned using uncooperative relativistic GAN.
复制标题
DOI：
10.1016/j.media.2022.102514
复制
发表时间：
2022-08
影响因子：
10.9
通讯作者：
Kamen, Ali
中科院分区：
文献类型：
--
作者：
Reaungamornrat, Sureerat;Sari, Hasan;Catana, Ciprian;Kamen, Ali
关键词：
由于人们对MR引导的放射治疗越来越感兴趣以及正电子发射断层扫描（PET）MR混合系统的引入，最近提出了越来越多的用于从磁共振（MR）图像估计衰减系数图的方法。我们提出了一个深度网络集成，将随机二进制解剖编码器和成像模态变分自编码器，将图像潜在空间分解为模态不变的解剖特征空间和模态属性空间。该集成集成了模态调制解码器，以基于成像模态来归一化特征和图像强度。除了促进解开，架构促进非合作学习，提供能力，以保持解剖结构在跨模态重建。引入模态不变的结构一致性约束，进一步加强了解剖结构的忠实嵌入。为了提高合成模态的训练稳定性和保真度，在包含多尺度鉴别器的相对论生成对抗框架中训练集成。对计算机断层扫描（CT）和MR骨盆数据集进行先验和网络架构分析以及性能验证。所提出的方法表现出对强度不均匀性的鲁棒性，改进了组织类别区分，并提供了Hounsfield单位的合成CT，与最先进的方法相比，具有跨切片的强度一致和平滑，在324张图像上提供了1.28的中值归一化互信息，0.97的归一化互相关和0.59的梯度互相关。
Growing number of methods for attenuation-coefficient map estimation from magnetic resonance (MR) images have recently been proposed because of the increasing interest in MR-guided radiotherapy and the introduction of positron emission tomography (PET) MR hybrid systems. We propose a deep-network ensemble incorporating stochastic-binary-anatomical encoders and imaging-modality variational autoencoders, to disentangle image-latent spaces into a space of modality-invariant anatomical features and spaces of modality attributes. The ensemble integrates modality-modulated decoders to normalize features and image intensities based on imaging modality. Besides promoting disentanglement, the architecture fosters uncooperative learning, offering ability to maintain anatomical structure in a cross-modality reconstruction. Introduction of a modality-invariant structural consistency constraint further enforces faithful embedding of anatomy. To improve training stability and fidelity of synthesized modalities, the ensemble is trained in a relativistic generative adversarial framework incorporating multiscale discriminators. Analyses of priors and network architectures as well as performance validation were performed on computed tomography (CT) and MR pelvis datasets. The proposed method demonstrated robustness against intensity inhomogeneity, improved tissue-class differentiation, and offered synthetic CT in Hounsfield units with intensities consistent and smooth across slices compared to the state-of-the-art approaches, offering median normalized mutual information of 1.28, normalized cross correlation of 0.97, and gradient cross correlation of 0.59 over 324 images.
影响因子：
3.5
作者：
Reaungamornrat S;De Silva T;Uneri A;Goerres J;Jacobson M;Ketcha M;Vogt S;Kleinszig G;Khanna AJ;Wolinsky JP;Prince JL;Siewerdsen JH
通讯作者：
Siewerdsen JH
影响因子：
5.7
作者：
Juttukonda MR;Mersereau BG;Chen Y;Su Y;Rubin BG;Benzinger TLS;Lalush DS;An H
通讯作者：
An H