Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation.
Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation.
复制标题
使用周期一致的对抗网络和评估来模拟逼真的胎儿神经超声图像的外观和生长变化。
DOI：
10.1117/1.jmi.7.5.057001
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Xu Y
中科院分区：
文献类型：
--
作者：
Xu Y
目的：我们提出了一种原始的方法，用于模拟逼真的胎儿神经超声图像，特别是从中期妊娠图像生成晚期妊娠超声图像。我们的方法是使用非配对数据开发的，因为没有配对数据。我们还报告了原始见解的一般外观之间的差异中期和晚期胎儿头部经心室（TV）plane images.Approach：我们设计了一个周期一致的对抗网络（Cycle-GAN）来模拟视觉上逼真的晚期图像从未配对的中期和晚期超声图像。模拟真实性由经验丰富的超声医师进行定性评估，他们对真实的和模拟图像进行盲分级。此外，还进行了定量评估，其中验证的基于深度学习的图像识别算法（ScanNav®）作为专家参考，允许自动分析和高效比较数百张真实的和模拟图像。结果：定性评估表明，人类专家无法区分真实的和模拟的妊娠晚期扫描图像之间的差异。84.2%的模拟孕晚期图像与真实的孕晚期图像不能区分。作为定量基线，在3000张图像上，通过ScanNav®计算了真实的中期和真实的晚期扫描之间脉络膜、CSP和中线镰的可见度下降，发现分别为72.5%、61.5%和67%。真实的孕中期和模拟孕晚期之间相同结构的可见性下降分别为77.5%、57.7%和56.2%。因此，认为真实的和模拟的妊娠晚期图像在视觉上彼此相似。我们的评估还表明，传统GAN的孕晚期模拟更容易区分，结构的可见性下降比我们提出的方法更小。结论：结果证实，使用修改后的Cycle-GAN从孕中期图像模拟逼真的孕晚期图像是可能的，这可能对孕晚期扫描有限但可以获得充足的孕中期图像的深度学习研究人员有用。我们还展示了令人信服的模拟改进，定性和定量，使用循环GAN方法相比，传统的GAN。最后，使用基于机器学习的参考（在ScanNav®案例中）进行大规模定量图像分析评估也是我们所知的第一次。
Purpose:We present an original method for simulating realistic fetal neurosonography images specifically generating third-trimester pregnancy ultrasound images from second-trimester images. Our method was developed using unpaired data, as pairwise data were not available. We also report original insights on the general appearance differences between second- and third-trimester fetal head transventricular (TV) plane images.Approach:We design a cycle-consistent adversarial network (Cycle-GAN) to simulate visually realistic third-trimester images from unpaired second- and third-trimester ultrasound images. Simulation realism is evaluated qualitatively by experienced sonographers who blindly graded real and simulated images. A quantitative evaluation is also performed whereby a validated deep-learning-based image recognition algorithm (ScanNav®) acts as the expert reference to allow hundreds of real and simulated images to be automatically analyzed and compared efficiently.Results:Qualitative evaluation shows that the human expert cannot tell the difference between real and simulated third-trimester scan images. 84.2% of the simulated third-trimester images could not be distinguished from the real third-trimester images. As a quantitative baseline, on 3000 images, the visibility drop of the choroid, CSP, and mid-line falx between real second- and real third-trimester scans was computed by ScanNav®and found to be 72.5%, 61.5%, and 67%, respectively. The visibility drop of the same structures between real second-trimester and simulated third-trimester was found to be 77.5%, 57.7%, and 56.2%, respectively. Therefore, the real and simulated third-trimester images were consider to be visually similar to each other. Our evaluation also shows that the third-trimester simulation of a conventional GAN is much easier to distinguish, and the visibility drop of the structures is smaller than our proposed method.Conclusions:The results confirm that it is possible to simulate realistic third-trimester images from second-trimester images using a modified Cycle-GAN, which may be useful for deep learning researchers with a restricted availability of third-trimester scans but with access to ample second trimester images. We also show convincing simulation improvements, both qualitatively and quantitatively, using the Cycle-GAN method compared with a conventional GAN. Finally, the use of a machine learning-based reference (in the case ScanNav®) for large-scale quantitative image analysis evaluation is also a first to our knowledge.