L-former: a lightweight transformer for realistic medical image generation and its application to super-resolution
L-former: a lightweight transformer for realistic medical image generation and its application to super-resolution
复制标题
DOI：
10.1117/12.2653776
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Tong Zheng;H. Oda;Y. Hayashi;Shota Nakamura;M. Mori;H. Takabatake;H. Natori;M. Oda;K. Mori
中科院分区：
文献类型：
--
作者：
Tong Zheng;H. Oda;Y. Hayashi;Shota Nakamura;M. Mori;H. Takabatake;H. Natori;M. Oda;K. Mori
数据增强和领域自适应等医学图像分析方法需要大量的真实医学图像。利用机器学习生成逼真的医学图像是一种可行的方法。我们提出了L-former，一种用于真实医学图像生成的轻量级变压器。与现有的生成对抗网络（gan）相比，L-former可以生成更可靠、更逼真的医学图像。同时，L-former不像传统的基于transformer的生成模型那样消耗高的计算成本。L-former在浅层使用Transformers生成低分辨率特征向量，在深层使用卷积神经网络生成高分辨率的真实医学图像。实验结果表明，L-former在两个数据集上的FID得分分别为33.79分和76.85分，优于传统gan。我们进一步进行了下游研究，利用L-former生成的图像执行超分辨率任务。27.87的高PSNR分数证明了L-former能够生成可靠的超分辨率图像，显示了其在医学诊断中的应用潜力。
Medical image analysis approaches such as data augmentation and domain adaption need huge amounts of realistic medical images. Generating realistic medical images by machine learning is a feasible approach. We propose L-former, a lightweight Transformer for realistic medical image generation. L-former can generate more reliable and realistic medical images than recent generative adversarial networks (GANs). Meanwhile, L-former does not consume as high computational cost as conventional Transformer-based generative models. L-former uses Transformers to generate low-resolution feature vectors at shallow layers, and uses convolutional neural networks to generate high-resolution realistic medical images at deep layers. Experimental results showed that L-former outperformed conventional GANs by FID scores 33.79 and 76.85 on two datasets, respectively. We further conducted a downstream study by using the images generated by L-former to perform a super-resolution task. A high PSNR score of 27.87 proved L-former’s ability to generate reliable images for super-resolution and showed its potential for applications in medical diagnosis.