Prediction of Immunotherapy Response with Geometric Deep Learning in Medical Imaging

癌症患者的治疗选择有限。免疫疗法是一种很有前途的策略，它利用免疫系统的主要功能来清除体内的癌细胞，显示出治愈癌症的长期益处。然而，临床医生仍然没有可靠的成像生物标志物，可以在治疗前可靠地识别对免疫疗法有反应的患者。图像生物标志物可以作为一种非侵入性的替代来表征肿瘤的空间异质性，但缺乏预测治疗结果的统计框架。图或流形上的深度学习，称为几何深度学习，最近在医学成像问题中引起了人们的兴趣，特别是在肿瘤学和免疫治疗中。他们展示了处理非欧几里德域的潜力，同时解决了有限和不平衡数据集等几个挑战。 全球目标是基于几何深度学习中的新概念，开发用于免疫治疗反应的图像驱动预测工具。所提出的方法将综合复杂和多模态的来源（CT图像和临床数据）描述的歧视性的图形，捕捉潜在的趋势，从纵向患者数据集。它还将允许解释深层特征，以捕获药物对结果的影响，以及恢复特定于免疫表型的时空潜在表征。为了实现这一全局目标，主要目标是：（1）开发一种用流形正则化深度神经网络训练的判别图结构，以预测有反应的患者;（2）用无监督域自适应来参数化潜在空间的拓扑结构，以量化对结果的治疗效果;（3）利用黎曼几何和递归网络中的概念，从时空深层特征创建治疗反应的预测模型;（4）在癌症患者数据集上应用并验证所提出的方法，将预测结果与实际结果进行比较。基于我们最近在几何深度学习方面的工作，该提案的贡献在于一个预测平台，该平台生成代表性知识，以发现免疫系统反应和肿瘤演变的预测生物标志物，用于诊断和治疗。这项研究提案将对计算医学成像产生重大影响，因为几何深度学习框架将允许揭示癌症表征的特定领域关系。它将合成纵向采集的大型异质医学图像集，并提供癌性肿瘤演变的全面画像。最后，它将提出新的功能，将复杂结构分解为联合子集，并适应通过深度学习获得的不同流形拓扑。从临床角度来看，这些发展将为癌症的早期检测和治疗结果的预测提供见解和指示，以及医学上可靠的预测工具。

Cancer patients have limited treatment options. Immunotherapy is a promising strategy using the primary function of the immune system to rid the body cancer cells, showing long-term benefits for curing cancer. However, clinicians still do not have a reliable imaging biomarker that can reliably identify patients responding to immunotherapy prior to treatment. Image biomarkers can serve as a non-invasive alternative to characterize the spatial heterogeneity of tumors, but lack the statistical framework to predict therapy outcomes. Deep learning on graphs or manifolds, denoted as geometric deep learning, has recently gained interest for medical imaging problems, specifically in oncology and immunotherapy. They demonstrated the potential to process non-Euclidean domains, while addressing several challenges such as limited and unbalanced datasets.  The global objective is to develop image-driven predictive tools for immunotherapy response, based on novel concepts in geometric deep learning. The proposed methods will synthesize complex and multi-modal sources (CT images and clinical data) described with discriminative graphs, capturing underlying trends from longitudinal patient datasets. It will also allow the interpretation of deep features to capture drug effects on outcomes, as well recover spatiotemporal latent representations specific to immune phenotypes. To achieve this global objective, the main objectives are: (1) Develop a discriminative graph structure trained with a manifold-regularized deep neural network to predict responsive patients; (2) Parameterize the topology of the latent space with unsupervised domain adaptation to quantify therapeutic effects on outcomes; (3) Exploit concepts in Riemannian geometry and recurrent networks to create a prediction model for treatment response from spatiotemporal deep features; (4) Apply and validate the proposed methods on cancer patient datasets, comparing predicted results with actual outcomes. Based on our recent work on geometric deep learning, the proposal's contributions lie in a predictive platform generating representative knowledge to discover predictive biomarkers of the immune system response and tumor evolution, aimed for diagnosis and therapy. This research proposal will have a significant impact in computational medical imaging, as the geometric deep learning framework will allow to uncover domain specific relationships for cancer characterization. It will synthesize large heterogeneous sets of medical images acquired longitudinally and provide a comprehensive portrait of cancerous tumor evolution. Finally, it will propose new functionalities to disentangle complex structures into joint subsets, as well as adapting to different manifold topologies obtained with deep learning. From a clinical perspective, these developments will contribute insights and indications for early detection of cancer and predictors of therapy outcomes, as well as reliable forecasting tools in medicine.