ICF: NIRG: HistoMaps: Stain agnostic feature representations to identify clinically relevant traits in the tumour microenvironment

在计算病理学的早期，算法主要集中在分割和识别日常临床实践中病理学家感兴趣的对象，如细胞核、腺体、导管、血管等。这一概念是为了帮助病理学家识别在整个幻灯片图像(WSI)中难以观察到的癌症组织的巨大景观。基于深度学习的现代CPATH算法的出现发现，由于无意中的盲目，人们通常会忽略一些隐藏的特征。因此，CPATH已经超越了对整个幻灯片图像(WSI)中单个模式的识别和分类，转向了WSI级别或病例级别的诊断、突变和治疗反应预测，发现了新的形态模式、组织表型，在某些情况下甚至超过了病理学家的表现。另一方面，DL算法通常被认为是一个黑匣子，因为学习的特征缺乏可解释性，这使得理解不同疾病的生物学变得困难。其中一个原因是我们无法分析肿瘤微环境(TME)中的巨大景观，在分析之前，WSIS被划分为小块，这是因为分析来自不同斑点和模式的图像所需的硬件限制和复杂的DL架构。挑战在于包含癌症图景的信息社会世界首脑会议的10亿像素大小，这一方面迫使人们进行探索，但同时也面临着技术挑战。由于肿瘤的异质性，这些小斑块通常不能代表WSIS。因此，我们需要开发技术来分析WSIS，而不需要将它们分成更小的块，保持空间信息的完整性。这不仅可以克服肿瘤的异质性限制，还有助于识别与患者预后和其他临床变量相关的异质性区域和嵌入的空间关系。这些技术应该能够克服硬件的实际限制，不受输入染色的影响，并且应该能够帮助解释和对TME的生物学理解。算法的局限性目前正在通过WSI级别的弱监督标签或压缩表示来解决。这些方法有一些主要的缺点，例如，这些方法丢弃了在压缩过程中合并临床重要区域中的细胞间相互作用所需的基本空间信息，并且主要集中在疾病的识别或分类在子类别中，其中DL模型被视为黑盒。在细胞水平上对TME的分析对于理解肿瘤异质性起重要作用的癌症发生机制很重要。多路免疫荧光(MxIF)图像为同一组织切片上的单个细胞亚型提供了额外的数据，这在现有的Brightfield方法中是不可能的。全载玻片荧光成像技术的最新进展使WSIS能够用多个标记物进行扫描。因此，我们需要染色和形态不可知的方法来分析WSIS，而不会丢失细胞水平的空间信息，这样就可以挖掘丰富的数据来更好地了解癌症。我们建议在现有技术的基础上，利用提取的信息在整个幻灯片图像层面上了解TME交互作用。在这个项目中，我们将开发染色不可知技术来分析和识别整个幻灯片图像(WSIS)中的模式，通过创建可与生物意义和临床相关参数(即突变、生存和治疗反应)直接相关的组织地图，将组织学景观与临床变量联系起来，以更好地了解癌症，帮助肿瘤学家在治疗干预措施上做出明智的决定，并帮助制药公司开发新的靶点。

In early years of computational pathology, the algorithms were mainly focussed on segmentation and identification of objects such as nuclei, glands, ducts, vessels, and other patterns which are of interest to pathologists in every day clinical practice. The concept was to assist pathologists in identifying patterns which are difficult to eyeball over the huge landscape of cancer tissue in a whole slide image (WSI). The advent of modern CPath algorithms based on deep learning (DL) found that there are hidden features which humans usually ignore due to inattentional blindness. Therefore, CPath has moved beyond identification and classification of individual patterns within a whole slide image (WSI) towards WSI-level or case-level diagnosis, mutation and therapeutic response prediction discovering new morphological patterns, tissue phenotypes, even surpassing pathologist performance in some cases. On the other hand, DL algorithms are usually considered to be a black box due to lack of interpretability of the learnt features which makes it difficult to understand the biology of different diseases. One of the reasons is our inability to analyse huge landscapes in the tumour microenvironment (TME) where the WSIs are divided into small patches before analysis due to hardware limitations and complex DL architectures required for the analysis of images from different stains and modalities. The challenge is the gigapixel size of the WSIs containing the landscape of cancer which on one hand compels exploration but at the same time faces technological challenges. Due to tumour heterogeneity these small patches are usually not representative of the WSIs. Therefore, we need to develop techniques which can analyse WSIs without dividing them into smaller patches keeping the spatial information intact. This not only allows to overcome tumour heterogeneity limitations but helps in identifying heterogenous regions and embedded spatial relationships linked to patient outcome and other clinical variables. These techniques should be able to overcome the practical limitations of the hardware, invariant to the input stains and should be able to help with interpretability and biological understanding of the TME. The algorithmic limitations are currently being tackled by WSI-level weakly supervised labels or compressed representations. These approaches have some major drawbacks e.g., these approaches discard the essential spatial information required to incorporate cell-to-cell interactions in clinically significant regions during compression and are focussed mostly on identification or classification of disease into sub-categories where the DL model is treated as a black box. Analysis of TME at the cellular level is important to understand mechanisms in cancer where tumour heterogeneity plays a significant role. Multiplexed Immunofluorescence (MxIF) images provide additional data to subtype individual cells on the same tissue section which is not currently possible with existing brightfield approaches. There have been recent advances in whole slide image fluorescence imaging which allow scanning of WSIs with multiple markers. Therefore, we need stain and modality agnostic approaches which can analyse WSIs without losing spatial information at the cellular level so the rich data can be mined for better understanding of cancer. We propose to build on existing technology and utilise the extracted information to understand TME interactions at the whole slide image level. In this project, we will develop stain agnostic techniques to analyse and identify patterns in whole slide images (WSIs) by creating HistoMaps which can be directly related to biologically meaningful and clinically relevant parameters i.e., mutations, survival and response to therapy linking histology landscapes to clinical variables for better understanding of cancer helping oncologists to make informed decisions on therapeutic interventions and assisting pharma to develop new targets.