Seamless Virtual Whole Slide Image Synthesis and Validation Using Perceptual Embedding Consistency
Seamless Virtual Whole Slide Image Synthesis and Validation Using Perceptual Embedding Consistency
复制标题
DOI：
10.1109/jbhi.2020.2975151
复制
发表时间：
2021-02-01
影响因子：
7.7
通讯作者：
Klaiman, Eldad
中科院分区：
文献类型：
--
作者：
Lahiani, Amal;Klaman, Irina;Klaiman, Eldad
染色虚拟化是一种越来越受关注的数字病理学应用，允许模拟染色组织图像，从而节省实验室和组织资源。由于生成对抗网络（GANs）的成功和无监督学习的进步，无监督风格迁移gan已经成功地用于生成逼真的、有临床意义的和可解释的图像。大尺寸的高分辨率全幻灯片图像（wsi）带来了额外的计算挑战。这使得在深度学习网络的训练和推理过程中需要平铺处理。实例规范化在风格迁移GAN应用中具有实质性的积极作用，但对于平铺推理，它倾向于在重构的wsi中引起平铺伪影。在本文中，我们提出了一种新的感知嵌入一致性（PEC）损失，迫使网络在潜在空间中学习颜色、对比度和亮度不变特征，从而大大减少了上述平铺伪影。我们的方法可以更加无缝地重建虚拟wsi。我们通过将虚拟生成的图像与相应的连续真实染色图像进行比较，定量地验证我们的方法。我们将我们的结果与最先进的无监督风格转移方法和从连续的真实染色组织幻灯片图像中获得的措施进行比较。我们通过比较模型对颜色、对比度和亮度扰动的鲁棒性以及瓶颈嵌入的可视化来证明我们关于PEC损失影响的假设。我们通过测量瓶颈特征图对不同扰动的敏感性并将其用于肿瘤分割任务来验证瓶颈特征图的鲁棒性。此外，我们建议通过比较两名病理学家对真实和虚拟瓷砖的解释以及病理学家之间的一致性来初步验证虚拟染色的应用。
Stain virtualization is an application with growing interest in digital pathology allowing simulation of stained tissue images thus saving lab and tissue resources. Thanks to the success of Generative Adversarial Networks (GANs) and the progress of unsupervised learning, unsupervised style transfer GANs have been successfully used to generate realistic, clinically meaningful and interpretable images. The large size of high resolution Whole Slide Images (WSIs) presents an additional computational challenge. This makes tilewise processing necessary during training and inference of deep learning networks. Instance normalization has a substantial positive effect in style transfer GAN applications but with tilewise inference, it has the tendency to cause a tiling artifact in reconstructed WSIs. In this paper we propose a novel perceptual embedding consistency (PEC) loss forcing the network to learn color, contrast and brightness invariant features in the latent space and hence substantially reducing the aforementioned tiling artifact. Our approach results in more seamless reconstruction of the virtual WSIs. We validate our method quantitatively by comparing the virtually generated images to their corresponding consecutive real stained images. We compare our results to state-of-the-art unsupervised style transfer methods and to the measures obtained from consecutive real stained tissue slide images. We demonstrate our hypothesis about effect of the PEC loss by comparing model robustness to color, contrast and brightness perturbations and visualizing bottleneck embeddings. We validate the robustness of the bottleneck feature maps by measuring their sensitivity to the different perturbations and using them in a tumor segmentation task. Additionally, we propose a preliminary validation of the virtual staining application by comparing interpretation of 2 pathologists on real and virtual tiles and inter-pathologist agreement.