Heterogeneous data fusion and machine learning for image understanding in lung cancer

肺癌仍然是全世界癌症死亡的最常见原因。对于肿瘤很小（小于5厘米）且未扩散到身体其他部位的早期非小细胞肺癌患者，标准治疗方法是手术或高剂量放射治疗。然而，即使这些癌症在早期被诊断出来，多达一半的患者在治疗后可能会复发，即癌症在同一部位或身体的其他部位复发。肺癌的主要问题之一是确定哪些患者在治疗后会被治愈。为了解决这个问题，本研究建议开发一种新的软件工具，以帮助医生确定哪些患者在治疗后复发的风险更高。在治疗之前，患者接受影像学检查以确定其疾病的程度，包括计算机断层扫描（CT）和位置发射断层扫描（PET）。然而，医生通常只在CT上测量肿瘤的直径，并在PET上寻找癌症扩散的区域。我们建议开发一种人工智能计算机系统，以帮助医生从这些医学图像中提取更多信息。人工智能的一个新领域，被称为深度学习，是一种人工神经网络，它是一种模仿生物神经元（如大脑中的神经元）结构和功能的软件程序。深度学习在许多医学领域显示出前景，包括理解成像数据。我们将开发基于深度学习的人工智能软件系统，整合医学影像和非影像患者数据，预测哪些患者治疗失败的风险更高。深度学习系统可以提取图像中的细微特征，这些特征可能是医生肉眼看不到的，并将其与其他患者信息结合起来。该模型将整合多模式和多尺度信息，包括三维医学成像数据（CT和PET）、临床参数（如年龄、吸烟史）、血液参数和肿瘤基因组信息。该软件系统将整合有关患者的多个信息来源，并为医生提供患者的预后，或标准治疗将治愈患者癌症的概率。我们还将首次开发一种新颖的图形用户界面，将这些信息可视化并显示给医生。总体而言，该研究项目中开发的软件工具将基于不同类型的肺部成像数据以及患者临床、血液和基因组信息的整合，实现准确的计算机辅助预后。这种非侵入性和廉价的软件工具将允许更好的肺癌预后特征，可以帮助医生识别复发风险较高的患者，以指示更积极或个性化的治疗方案。

Lung cancer remains the most common cause of cancer death worldwide. For patients with early-stage non-small cell lung cancer, where the tumour is small (less than 5 cm) and has not spread to other parts of the body, standard treatment is either surgery or high-dose radiation therapy. However, even when these cancers are diagnosed at an early-stage, up to half of patients may develop a recurrence after treatment, in which the cancer returns at the same spot or somewhere else in the body. One of the major problems with lung cancer is determining which patients will be cured of their disease following treatment. To solve this problem, this research proposes to develop a novel software tool to aid physicians in determining which patients are at a higher risk of recurrence following treatment. Prior to treatment patients receive imaging to determine the extent of their disease, including computed tomography (CT) and position emission tomography (PET). However, physicians typically only measure the diameter of the tumour on CT and look for areas where the cancer has spread on PET. We propose to develop an artificially intelligent computer system to help physicians extract more information from these medical images. A new area of artificial intelligence, known as deep learning is a type of artificial neural network, which is a software program that mimics the structure and function of biological neurons, such as those in the brain. Deep learning has shown promise in many areas of medicine, including understanding imaging data. We will develop a deep learning based artificial intelligence software system to integrate medical imaging and the non-imaging patient data to predict which patients are at a higher risk of treatment failure. A deep learning system can extract subtle features within the image, that may not be visible by the physician's eye, and combine it with other patient information. This model will integrate multi-modal and multi-scale information, including 3-dimensional medical imaging data (CT and PET), clinical parameters (e.g., age, smoking history), blood parameters, and tumour genomic information. This software system will integrate multiple sources of information about a patient and provide the physician with a prognosis for the patient, or a probability that the standard treatment will cure the patient's cancer. We will also develop, for the first time, a novel graphical user interface to visualize and display this information to the physician. Overall, the software tool developed within this research program will enable accurate computer-aided prognosis based on different types of lung imaging data and the integration of clinical, blood, and genomic information about a patient. This non-invasive and inexpensive software tool will allow for better prognostic characterization of lung cancer that can help physicians in identifying patients at higher risk of recurrence for indicating more aggressive or personalized treatment options.