Deep learning based registration of serial whole-slide histopathology images in different stains.
Deep learning based registration of serial whole-slide histopathology images in different stains.
复制标题
DOI：
10.1016/j.jpi.2023.100311
复制
发表时间：
2023
影响因子：
--
通讯作者：
Kong, Jun
中科院分区：
文献类型：
--
作者：
Roy, Mousumi;Wang, Fusheng;Teodoro, George;Bhattarai, Shristi;Bhargava, Mahak;Rekha, T Subbanna;Aneja, Ritu;Kong, Jun
对于常规病理诊断和基于成像的生物医学研究，全幻灯片图像 (WSI) 分析在很大程度上仅限于二维组织图像空间。为了获得更明确的组织表示以支持高分辨率空间和综合分析，至关重要的是，将这种基于组织的研究扩展到 3D 组织空间，并在不同染色中使用空间对齐的连续组织 WSI，例如苏木精和曙红 (H&E) 以及免疫组织化学 (IHC) 生物标志物。然而，这种WSI配准在技术上面临着巨大的图像尺度、复杂的组织学结构变化以及不同染色中组织外观的显着差异等挑战。本研究的目标是配准来自多染色组织病理学全幻灯片图像块的连续切片。我们提出了一种新颖的基于翻译的深度学习配准网络 CGNReg，它可以在空间上对齐 H&E 和 IHC 生物标志物染色的序列 WSI，而无需模型训练的先验变形信息。首先，通过强大的图像合成算法从 H&E 载玻片生成合成 IHC 图像。接下来，通过具有多尺度可变形矢量场和联合损失优化的全卷积网络来配准合成图像和真实 IHC 图像。我们以全图像分辨率进行配准，在结果中保留组织细节。通过对 76 名乳腺癌患者的数据集进行评估，每位患者进行 1 个 H&E 和 2 个 IHC 系列 WSI，与我们评估中的多个最先进系统相比，CGNReg 表现出有希望的性能。我们的结果表明，CGNReg 可以通过不同染色中的连续 WSI 产生有希望的配准结果，从而实现基于 3D 组织的综合生物医学研究。来自不同染色中空间对齐的连续组织 WSI 的 3D 组织空间。 WSI 配准挑战：图像尺寸大、组织外观差异。基于翻译的深度学习模型 CGNReg 用于空间对齐多染色序列 WSI。以全图像分辨率进行配准，保留组织细节。 CGNReg 可以实现基于组织的综合 3D 生物医学研究。
For routine pathology diagnosis and imaging-based biomedical research, Whole-slide image (WSI) analyses have been largely limited to a 2D tissue image space. For a more definitive tissue representation to support fine-resolution spatial and integrative analyses, it is critical to extend such tissue-based investigations to a 3D tissue space with spatially aligned serial tissue WSIs in different stains, such as Hematoxylin and Eosin (H&E) and Immunohistochemistry (IHC) biomarkers. However, such WSI registration is technically challenged by the overwhelming image scale, the complex histology structure change, and the significant difference in tissue appearances in different stains. The goal of this study is to register serial sections from multi-stain histopathology whole-slide image blocks. We propose a novel translation-based deep learning registration network CGNReg that spatially aligns serial WSIs stained in H&E and by IHC biomarkers without prior deformation information for the model training. First, synthetic IHC images are produced from H&E slides through a robust image synthesis algorithm. Next, the synthetic and the real IHC images are registered through a Fully Convolutional Network with multi-scaled deformable vector fields and a joint loss optimization. We perform the registration at the full image resolution, retaining the tissue details in the results. Evaluated with a dataset of 76 breast cancer patients with 1 H&E and 2 IHC serial WSIs for each patient, CGNReg presents promising performance as compared with multiple state-of-the-art systems in our evaluation. Our results suggest that CGNReg can produce promising registration results with serial WSIs in different stains, enabling integrative 3D tissue-based biomedical investigations. 3D tissue space from spatially aligned serial tissue WSIs in different stains. WSI registration challenges: large image scale, difference in tissue appearances. Translation-based DL model CGNReg to spatially align multi-stained serial WSIs. Registration at the full image resolution retaining the tissue details. CGNReg can enable integrative 3D tissue-based biomedical investigations.