Multimodal image translation via deep learning inference model trained in video domain.
Multimodal image translation via deep learning inference model trained in video domain.
复制标题
通过在视频领域训练的深度学习推理模型进行多模态图像翻译
DOI：
10.1186/s12880-022-00854-x
复制
发表时间：
2022-07-14
影响因子：
2.7
通讯作者：
Hu, Weigang
中科院分区：
文献类型：
--
作者：
Fan, Jiawei;Liu, Zhiqiang;Yang, Dong;Qiao, Jian;Zhao, Jun;Wang, Jiazhou;Hu, Weigang
目前的医学图像翻译都是在图像域实现的。考虑到医学图像的获取本质上是一个时间连续的过程，我们试图通过在视频域训练的深度学习来开发一种新的图像翻译框架，用于从锥束计算机断层扫描(CBCT)图像生成合成的CT图像。为了进行概念验证，收集了100名患者的CBCT和CT图像，以证明所提出框架的可行性和可靠性。CBCT和CT图像进一步配准为配对样本，并作为有监督模型训练的输入数据。采用基于条件GAN网络的vid2vid框架，通过精心设计的生成器、鉴别器和新的时空学习目标，实现了CBCT-CT图像在视频域的平移。对10个新测试患者的真实和合成CT图像计算了平均绝对误差(MAE)、峰值信噪比(PSNR)、归一化互相关(NCC)和结构相似度(SSIM)等4个评价指标，以说明模型的性能。MAE、PSNR、NCC和SSIM四个评价指标的平均值分别为23.27 ± 5.53、32.67 ± 1.98、0.99 ± 0.0059和0.97 ± 0.028。真实CT图像和合成CT图像之间的大部分像素方向Hounsfield单元值差异在50以内。合成CT图像与真实CT图像具有很好的一致性，与CBCT图像相比，图像质量得到了改善，噪声和伪影也有所降低。我们提出了一种基于深度学习的方法来处理视频域的医学图像翻译问题。虽然CBCT-CT图像翻译验证了该框架的可行性和可靠性，但该框架可以很容易地扩展到其他类型的医学图像。目前的研究结果表明，这是一种很有前途的方法，可能会为医学图像翻译研究开辟一条新的道路。
Current medical image translation is implemented in the image domain. Considering the medical image acquisition is essentially a temporally continuous process, we attempt to develop a novel image translation framework via deep learning trained in video domain for generating synthesized computed tomography (CT) images from cone-beam computed tomography (CBCT) images. For a proof-of-concept demonstration, CBCT and CT images from 100 patients were collected to demonstrate the feasibility and reliability of the proposed framework. The CBCT and CT images were further registered as paired samples and used as the input data for the supervised model training. A vid2vid framework based on the conditional GAN network, with carefully-designed generators, discriminators and a new spatio-temporal learning objective, was applied to realize the CBCT–CT image translation in the video domain. Four evaluation metrics, including mean absolute error (MAE), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC), and structural similarity (SSIM), were calculated on all the real and synthetic CT images from 10 new testing patients to illustrate the model performance. The average values for four evaluation metrics, including MAE, PSNR, NCC, and SSIM, are 23.27 ± 5.53, 32.67 ± 1.98, 0.99 ± 0.0059, and 0.97 ± 0.028, respectively. Most of the pixel-wise hounsfield units value differences between real and synthetic CT images are within 50. The synthetic CT images have great agreement with the real CT images and the image quality is improved with lower noise and artifacts compared with CBCT images. We developed a deep-learning-based approach to perform the medical image translation problem in the video domain. Although the feasibility and reliability of the proposed framework were demonstrated by CBCT–CT image translation, it can be easily extended to other types of medical images. The current results illustrate that it is a very promising method that may pave a new path for medical image translation research.
登录
查看更多内容
影响因子：
3.8
作者：
Liu Y;Lei Y;Wang T;Fu Y;Tang X;Curran WJ;Liu T;Patel P;Yang X
通讯作者：
Yang X
基于 MV CBCT 的合成 CT 生成，采用深度学习方法进行直肠癌适应性放射治疗
影响因子：
4.7
作者：
Zhao J;Chen Z;Wang J;Xia F;Peng J;Hu Y;Hu W;Zhang Z
通讯作者：
Zhang Z
基于深度学习技术预测三维剂量分布的自动治疗计划
影响因子：
3.8
作者：
Fan, Jiawei;Wang, Jiazhou;Hu, Weigang
通讯作者：
Hu, Weigang
影响因子：
10.9
作者：
Hering, Alessa;Haeger, Stephanie;Moltz, Jan;Lessmann, Nikolas;Heldmann, Stefan;van Ginneken, Bram
通讯作者：
van Ginneken, Bram
影响因子：
3.5
作者：
Liang, Xiao;Chen, Liyuan;Jiang, Steve
通讯作者：
Jiang, Steve