Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection
Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection
复制标题
DOI：
10.1109/access.2019.2947606
复制
发表时间：
2019-01-01
期刊：
影响因子：
3.9
通讯作者：
Hayashi, Hideaki
中科院分区：
文献类型：
--
作者：
Han, Changhee;Rundo, Leonardo;Hayashi, Hideaki
卷积神经网络（cnn）通过足够的带注释的训练数据实现了出色的计算机辅助诊断。然而，大多数医学成像数据集都是小而碎片化的。在这种情况下，生成对抗网络（GANs）可以合成逼真/多样化的附加训练图像来填补真实图像分布中的数据不足；研究人员通过使用噪声到图像（例如，随机噪声样本到各种病理图像）或图像到图像gan（例如，良性图像到恶性图像）增强数据来改进分类。然而，没有研究报告将噪声到图像和图像到图像的gan结合起来进一步提高性能。因此，为了最大化GAN组合的DA效果，我们提出了一种基于GAN的两步DA，分别生成和细化有/没有肿瘤的脑磁共振（MR）图像：（${i}$） GAN的渐进式生长（PGGANs），用于高分辨率MR图像生成的多阶段噪声到图像GAN，首先生成逼真/多样化的$256\ × 256$图像；（ii）多模态无监督图像到图像转换（MUNIT），结合GAN /变分自动编码器或SimGAN，使用da聚焦的GAN损失，进一步细化pggan生成的图像的纹理/形状，类似于真实图像。我们深入研究了基于cnn的肿瘤分类结果，同时考虑了预训练对ImageNet的影响，并丢弃了长相怪异的gan生成的图像。结果表明，当与经典DA结合时，我们的基于gan的两步DA在肿瘤检测（即灵敏度从93.67提高到97.48）和其他医学成像任务中都明显优于单独的经典DA。
Convolutional Neural Networks (CNNs) achieve excellent computer-assisted diagnosis with sufficient annotated training data. However, most medical imaging datasets are small and fragmented. In this context, Generative Adversarial Networks (GANs) can synthesize realistic/diverse additional training images to fill the data lack in the real image distribution; researchers have improved classification by augmenting data with noise-to-image (e.g., random noise samples to diverse pathological images) or image-to-image GANs (e.g., a benign image to a malignant one). Yet, no research has reported results combining noise-to-image and image-to-image GANs for further performance boost. Therefore, to maximize the DA effect with the GAN combinations, we propose a two-step GAN-based DA that generates and refines brain Magnetic Resonance (MR) images with/without tumors separately: (${i}$ ) Progressive Growing of GANs (PGGANs), multi-stage noise-to-image GAN for high-resolution MR image generation, first generates realistic/diverse $256\times 256$ images; (ii) Multimodal UNsupervised Image-to-image Translation (MUNIT) that combines GANs/Variational AutoEncoders or SimGAN that uses a DA-focused GAN loss, further refines the texture/shape of the PGGAN-generated images similarly to the real ones. We thoroughly investigate CNN-based tumor classification results, also considering the influence of pre-training on ImageNet and discarding weird-looking GAN-generated images. The results show that, when combined with classic DA, our two-step GAN-based DA can significantly outperform the classic DA alone, in tumor detection (i.e., boosting sensitivity 93.67 to 97.48) and also in other medical imaging tasks.