Training generative adversarial networks for optical property mapping using synthetic image data.
Training generative adversarial networks for optical property mapping using synthetic image data.
复制标题
DOI：
10.1364/boe.458554
复制
发表时间：
2022-10-01
影响因子：
3.4
通讯作者：
Gordon, G. S. D.
中科院分区：
文献类型：
--
作者：
Osman, A.;Crowley, J.;Gordon, G. S. D.
我们使用空间频域成像(SFDI)图像数据集对生成性对抗网络(GAN)进行了训练，以预测光学属性图(散射和吸收)，这些图像数据集是由免费的开源3D建模和渲染软件Blender合成生成的。利用Blder的灵活性，模拟了5个与疾病组织临床SFDI相关的模型：包含单一材料的平板样本、包含2种材料的平板样本、包含3种材料的平板样本、具有球形肿瘤的平板样本和具有球形肿瘤的圆柱状样本。最后一个案例特别相关，因为它代表了管状器官内的广域成像，例如胃肠道。在所有5种情况下，我们显示了GaN能够从单个SFDI图像中准确地重建光学属性，平均归一化误差范围为1.0-1.2%的吸收和1.1%-1.2%的散射，从而在视觉上改善了肿瘤球状结构的对比度。这与在实验数据上使用GANS获得的∼10%的吸收误差和∼10%的散射误差进行了比较。接下来，我们对我们合成训练的GaN进行双向交叉验证，用90%的合成数据和10%的实验数据重新训练，以鼓励结构域转移，用完全根据实验数据训练的GaN，观察视觉上准确的结果，吸收的误差为6.3%-10.3%，散射的误差为6.6%-11.9%。因此，我们经过综合训练的GaN与真实的实验样本高度相关，但提供了大量训练数据集、完美的地面实况以及测试真实成像几何形状的能力，例如在圆柱体内部，对于这些圆柱体，不存在传统的单次解调算法。在未来，我们期望领域适配或混合真实-合成数据集上的训练等技术的应用将为快速、准确地产生用于真实临床成像系统的光学属性图创造一个强大的工具。
We demonstrate the training of a generative adversarial network (GAN) for the prediction of optical property maps (scattering and absorption) using spatial frequency domain imaging (SFDI) image data sets that are generated synthetically with a free open-source 3D modelling and rendering software, Blender. The flexibility of Blender is exploited to simulate 5 models with real-life relevance to clinical SFDI of diseased tissue: flat samples containing a single material, flat samples containing 2 materials, flat samples containing 3 materials, flat samples with spheroidal tumours and cylindrical samples with spheroidal tumours. The last case is particularly relevant as it represents wide-field imaging inside a tubular organ e.g. the gastro-intestinal tract. In all 5 scenarios we show the GAN provides an accurate reconstruction of the optical properties from single SFDI images with a mean normalised error ranging from 1.0-1.2% for absorption and 1.1%-1.2% for scattering, resulting in visually improved contrast for tumour spheroid structures. This compares favourably with the ∼10% absorption error and ∼10% scattering error achieved using GANs on experimental SFDI data. Next, we perform a bi-directional cross-validation of our synthetically-trained GAN, retrained with 90% synthetic and 10% experimental data to encourage domain transfer, with a GAN trained fully on experimental data and observe visually accurate results with an error of 6.3%-10.3% for absorption and 6.6%-11.9% for scattering. Our synthetically trained GAN is therefore highly relevant to real experimental samples but provides the significant added benefits of large training datasets, perfect ground-truths and the ability to test realistic imaging geometries, e.g. inside cylinders, for which no conventional single-shot demodulation algorithms exist. In the future, we expect that the application of techniques such as domain adaptation or training on hybrid real-synthetic datasets will create a powerful tool for fast, accurate production of optical property maps for real clinical imaging systems.