Evaluation of 3D GANs for Lung Tissue Modelling in Pulmonary CT
Evaluation of 3D GANs for Lung Tissue Modelling in Pulmonary CT
复制标题
DOI：
10.59275/j.melba.2022-9e4b
复制
发表时间：
2022-08
期刊：
影响因子：
--
通讯作者：
S. Ellis;O. M. Manzanera;V. Baltatzis;Ibrahim Nawaz;A. Nair;L. L. Folgoc-L.;S. Desai;Ben Glocker;J. Schnabel
中科院分区：
文献类型：
--
作者：
S. Ellis;O. M. Manzanera;V. Baltatzis;Ibrahim Nawaz;A. Nair;L. L. Folgoc-L.;S. Desai;Ben Glocker;J. Schnabel
生成对抗网络 (GAN) 能够准确地对复杂、高维数据集（例如图像）的分布进行建模。这一特性使得高质量的 GAN 对于医学成像中的无监督异常检测非常有用。然而，训练数据集的差异（例如输出图像维度和语义上有意义的特征的出现）意味着来自自然图像处理领域的 GAN 模型可能无法“开箱即用”地用于医学成像应用，需要重新实现和重新评估。在这项工作中，我们调整并评估了三种 GAN 模型，以应用于肺部 CT 的 3D 健康图像块建模。据我们所知，这是第一次进行如此详细的评估。由于深度卷积 GAN (DCGAN)、styleGAN 和 bigGAN 架构在自然图像处理中的普遍性和高性能，因此被选中进行研究。我们训练这些方法的不同变体，并使用广泛使用的弗雷切起始距离（FID）评估它们的性能。此外，通过人类观察者研究评估了生成图像的质量，研究了网络建模 3D 特定领域特征的能力，并分析了 GAN 潜在空间的结构。结果表明，3D styleGAN 方法可生成具有有意义的 3D 结构的逼真图像，但会遭受模式崩溃的问题，必须在训练期间明确解决该问题以获得样本的多样性。相反，3D DCGAN 模型显示出更大的图像可变性能力，但代价是图像质量较差。 3D bigGAN 模型提供中等水平的图像质量，但最准确地对所选语义有意义的特征的分布进行建模。结果表明，未来的发展需要实现具有足够表征能力的 3D GAN，用于基于补丁的肺部 CT 异常检测，我们为未来的研究领域提供了建议，例如尝试其他架构和结合位置编码。
Generative adversarial networks (GANs) are able to model accurately the distribution of complex, high-dimensional datasets, for example images. This characteristic makes high-quality GANs useful for unsupervised anomaly detection in medical imaging. However, differences in training datasets such as output image dimensionality and appearance of semantically meaningful features mean that GAN models from the natural image processing domain may not work 'out-of-the-box' for medical imaging applications, necessitating re-implementation and re-evaluation. In this work we adapt and evaluate three GAN models to the application of modelling 3D healthy image patches for pulmonary CT. To the best of our knowledge, this is the first time that such a detailed evaluation has been performed. The deep convolutional GAN (DCGAN), styleGAN and the bigGAN architectures were selected for investigation due to their ubiquity and high performance in natural image processing. We train different variants of these methods and assess their performance using the widely used Frechet Inception Distance (FID). In addition, the quality of the generated images was evaluated by a human observer study, the ability of the networks to model 3D domain-specific features was investigated, and the structure of the GAN latent spaces was analysed. Results show that the 3D styleGAN approaches produce realistic-looking images with meaningful 3D structure, but suffer from mode collapse which must be explicitly addressed during training to obtain diversity in the samples. Conversely, the 3D DCGAN models show a greater capacity for image variability, but at the cost of poor-quality images. The 3D bigGAN models provide an intermediate level of image quality, but most accurately model the distribution of selected semantically meaningful features. The results suggest that future development is required to realise a 3D GAN with sufficient representational capacity for patch-based lung CT anomaly detection and we offer recommendations for future areas of research, such as experimenting with other architectures and incorporation of position-encoding.