ICF: NIRG: HistoMaps: Stain agnostic feature representations to identify clinically relevant traits in the tumour microenvironment

在计算病理学的早期，算法主要集中在分割和识别对象，如细胞核，腺体，导管，血管和其他模式，这是病理学家在日常临床实践中感兴趣的。这个概念是为了帮助病理学家在整个幻灯片图像（WSI）中识别难以在巨大的癌症组织景观上进行观察的模式。基于深度学习（DL）的现代CPath算法的出现发现，人类通常会由于无意视盲而忽略隐藏的特征。因此，CPath已经超越了整个载玻片图像（WSI）中个体模式的识别和分类，转向WSI级或病例级诊断、突变和治疗反应预测，发现新的形态学模式、组织表型，甚至在某些情况下超过病理学家的表现。另一方面，深度学习算法通常被认为是一个黑盒子，因为缺乏对学习特征的可解释性，这使得难以理解不同疾病的生物学。原因之一是我们无法分析肿瘤微环境（TME）中的巨大景观，其中由于硬件限制和分析不同染色剂和模态图像所需的复杂DL架构，WSI在分析之前被分成小块。挑战在于包含癌症景观的WSI的千兆像素大小，这一方面迫使探索，但同时面临技术挑战。由于肿瘤的异质性，这些小斑块通常不能代表WSI。因此，我们需要开发的技术，可以分析WSI，而不是将它们分成更小的补丁保持空间信息的完整性。这不仅允许克服肿瘤异质性限制，而且有助于识别异质性区域和与患者结果和其他临床变量相关的嵌入式空间关系。这些技术应该能够克服硬件的实际限制，对输入染色剂不变，并且应该能够帮助TME的可解释性和生物学理解。算法的局限性目前正在通过WSI级别的弱监督标签或压缩表示来解决。这些方法具有一些主要缺点，这些方法丢弃了在压缩过程中在临床显著区域中结合细胞-细胞相互作用所需的基本空间信息，并且主要集中于将疾病识别或分类为子类别，其中DL模型被视为黑盒。在细胞水平上分析TME对于理解肿瘤异质性起重要作用的癌症机制非常重要。多重免疫荧光（MxIF）图像提供了额外的数据，以在同一组织切片上对单个细胞进行分型，这是现有明场方法目前无法实现的。最近在全载玻片图像荧光成像方面取得了进展，其允许扫描具有多个标记物的WSI。因此，我们需要染色和模态不可知的方法，可以分析WSI而不会丢失细胞水平的空间信息，因此可以挖掘丰富的数据以更好地了解癌症。我们建议建立在现有技术的基础上，并利用提取的信息来了解TME在整个幻灯片图像级别的相互作用。在这个项目中，我们将开发染色不可知技术，通过创建HistoMaps来分析和识别全切片图像（WSI）中的模式，HistoMaps可以直接与具有生物学意义和临床相关的参数相关，即，突变、生存和对治疗的反应将组织学景观与临床变量联系起来，以更好地了解癌症，帮助肿瘤学家就治疗干预措施做出明智的决定，并协助制药公司开发新的靶点。

In early years of computational pathology, the algorithms were mainly focussed on segmentation and identification of objects such as nuclei, glands, ducts, vessels, and other patterns which are of interest to pathologists in every day clinical practice. The concept was to assist pathologists in identifying patterns which are difficult to eyeball over the huge landscape of cancer tissue in a whole slide image (WSI). The advent of modern CPath algorithms based on deep learning (DL) found that there are hidden features which humans usually ignore due to inattentional blindness. Therefore, CPath has moved beyond identification and classification of individual patterns within a whole slide image (WSI) towards WSI-level or case-level diagnosis, mutation and therapeutic response prediction discovering new morphological patterns, tissue phenotypes, even surpassing pathologist performance in some cases. On the other hand, DL algorithms are usually considered to be a black box due to lack of interpretability of the learnt features which makes it difficult to understand the biology of different diseases. One of the reasons is our inability to analyse huge landscapes in the tumour microenvironment (TME) where the WSIs are divided into small patches before analysis due to hardware limitations and complex DL architectures required for the analysis of images from different stains and modalities. The challenge is the gigapixel size of the WSIs containing the landscape of cancer which on one hand compels exploration but at the same time faces technological challenges. Due to tumour heterogeneity these small patches are usually not representative of the WSIs. Therefore, we need to develop techniques which can analyse WSIs without dividing them into smaller patches keeping the spatial information intact. This not only allows to overcome tumour heterogeneity limitations but helps in identifying heterogenous regions and embedded spatial relationships linked to patient outcome and other clinical variables. These techniques should be able to overcome the practical limitations of the hardware, invariant to the input stains and should be able to help with interpretability and biological understanding of the TME. The algorithmic limitations are currently being tackled by WSI-level weakly supervised labels or compressed representations. These approaches have some major drawbacks e.g., these approaches discard the essential spatial information required to incorporate cell-to-cell interactions in clinically significant regions during compression and are focussed mostly on identification or classification of disease into sub-categories where the DL model is treated as a black box. Analysis of TME at the cellular level is important to understand mechanisms in cancer where tumour heterogeneity plays a significant role. Multiplexed Immunofluorescence (MxIF) images provide additional data to subtype individual cells on the same tissue section which is not currently possible with existing brightfield approaches. There have been recent advances in whole slide image fluorescence imaging which allow scanning of WSIs with multiple markers. Therefore, we need stain and modality agnostic approaches which can analyse WSIs without losing spatial information at the cellular level so the rich data can be mined for better understanding of cancer. We propose to build on existing technology and utilise the extracted information to understand TME interactions at the whole slide image level. In this project, we will develop stain agnostic techniques to analyse and identify patterns in whole slide images (WSIs) by creating HistoMaps which can be directly related to biologically meaningful and clinically relevant parameters i.e., mutations, survival and response to therapy linking histology landscapes to clinical variables for better understanding of cancer helping oncologists to make informed decisions on therapeutic interventions and assisting pharma to develop new targets.