Multimodal MR Synthesis via Modality-Invariant Latent Representation.
Multimodal MR Synthesis via Modality-Invariant Latent Representation.
复制标题
DOI：
10.1109/tmi.2017.2764326
复制
发表时间：
2018-03
影响因子：
10.6
通讯作者：
Tsaftaris SA
中科院分区：
文献类型：
--
作者：
Chartsias A;Joyce T;Giuffrida MV;Tsaftaris SA
我们提出了一个多输入多输出的全卷积神经网络模型的MRI合成。该模型对缺失数据具有鲁棒性，因为它受益于但不需要额外的输入模式。该模型是端到端训练的，并学习将所有输入模态嵌入到共享模态不变的潜在空间中。这些潜在的表示，然后组合成一个单一的融合表示，这是转换成目标输出模态与学习解码器。我们避免了课程学习的需要，利用事实上，各种输入方式是高度相关的。我们还表明，通过结合分割掩模的信息，该模型既可以减少其误差，又可以生成具有合成病变的数据。我们在ISLES和BRATS数据集上评估了我们的模型，并证明了对单输入任务的最先进方法的统计学显着改进。当使用多个输入模态时，这种改进进一步增加，证明了学习公共潜在空间的好处，再次导致比当前最佳方法在统计上显著的改进。最后，我们展示了我们的方法对非头骨剥离的大脑图像，产生了统计上的显着改善，比以前的最佳方法。代码可在https://github.com/agis85/multimodal brain synthesis上公开获取。
We propose a multi-input multi-output fully convolutional neural network model for MRI synthesis. The model is robust to missing data, as it benefits from, but does not require, additional input modalities. The model is trained end-to-end, and learns to embed all input modalities into a shared modality-invariant latent space. These latent representations are then combined into a single fused representation, which is transformed into the target output modality with a learnt decoder. We avoid the need for curriculum learning by exploiting the fact that the various input modalities are highly correlated. We also show that by incorporating information from segmentation masks the model can both decrease its error and generate data with synthetic lesions. We evaluate our model on the ISLES and BRATS datasets and demonstrate statistically significant improvements over state-of-the-art methods for single input tasks. This improvement increases further when multiple input modalities are used, demonstrating the benefits of learning a common latent space, again resulting in a statistically significant improvement over the current best method. Lastly, we demonstrate our approach on non skull-stripped brain images, producing a statistically significant improvement over the previous best method. Code is made publicly available at https://github.com/agis85/multimodal brain synthesis.