Early Alzheimers Forecasting from Multimodal Data via Deep Transfer Learning, Evaluated on a Large-Scale Prospective Cohort Study

项目摘要 阿尔茨海默病是一种无法治愈的衰弱和退行性大脑疾病，影响着约580万人 在美国的人们。该项目将开发培训、调整和转移模型的技术 从多模式数据中早期检测阿尔茨海默病，包括遗传信息、大脑 核磁共振和认知测试，重点是在普通人群中筛查AD(即，根据 具有代表性的横断面前瞻性队列研究)。 我们将引入基于深度迁移学习的新技术来从大脑中提取表征 磁共振成像，适用于没有专家注释的前瞻性收集的数据。我们会 使用多模式深度学习将特征提取整合到端到端预测框架中。 这些方法将有助于对阿尔茨海默氏症的进展进行建模、监测和预测 疾病，其中核磁共振伴随着在不同粒度级别收集的临床信息。我们 将从预测AD演变的模型开始，该模型基于来自 阿尔茨海默病神经成像倡议(ADNI)研究。根据ADNI数据训练的模型通常依赖于 关于需要相当多领域的大脑磁共振成像的专门工程特征 知识和前处理，如果患者要获得 在医院做核磁共振扫描。因此，我们训练了基于CNN的模型，这些模型直接与大脑核磁共振一起工作， 经过优化，可捕捉ADNI中提供的工程功能的预测能力。我们整合了 具有使用深度学习提取抽象表征的预测模型的脑MRI网络 基于受试者在给定点的多模式信息的健康状态，包括 人口统计学、遗传信息(例如，载脂蛋白E基因)、认知测试分数和大脑核磁共振成像。这个 方法学习健康状态转换，以及如何将健康状态抽象映射到诊断。 一个重要的创新是以端到端的方式并入图像提取组件 在使用混合卷积层的框架中，由领域知识和 信息理论测量，从图像中提取不同的特征。 此外，我们还介绍了模型在收集的数据集之间无缝转换的方法 作为不同研究的一部分，记录的信息，包括临床测试、收集的图像和 受试者问卷在不同的研究队列中有所不同。这些方法缓解了由此带来的挑战 否则，通过使用融合的信号和抽象之间的映射来获取丰富多样的数据。 在这项研究结束时，我们将建立一个通用的预测框架，能够预测 阿尔茨海默氏症在症状出现前几年发病，这是一个惊人的进步，将使临床医生能够 确定新的预防策略，为阿尔茨海默氏症做好准备，而不是做出回应。

Project Summary Alzheimer's Disease, a debilitating and degenerative brain disease that has no cure, affects ~5.8 million people in the United States. This project will develop techniques to train, adapt and transfer models for the early detection of Alzheimer’s disease from multimodal data, including genetic information, brain MRIs and cognitive tests, with a focus on screening for AD in the general population (i.e., evaluated on a cross-sectional, prospective cohort study, representative of the populations). We will introduce new techniques, based on deep transfer learning, to extract representations from brain MRIs, applicable to prospectively collected data which is unaccompanied by expert annotations. We will incorporate the feature extraction in an end-to-end predictive framework using multimodal deep learning. Such methods will be useful for modeling, monitoring, and forecasting the progression of Alzheimer's disease, where MRIs accompany the clinical information collected at different levels of granularity. We will start with a model that predicts the evolution of AD, trained on multimodal longitudinal data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) study. Models trained on ADNI data typically rely on specialized engineered features from the brain MRIs requiring a considerable amount of domain knowledge and pre-processing and which would not be generally available if a patient were to obtain an MRI scan in the hospital. Thus, we train CNN-based models that work directly with brain MRIs, optimized to capture the predictive capabilities of the engineered features present in ADNI. We integrate the brain MRI network with a forecasting model that uses deep learning to extract abstract representations of the subjects' health status based on their multimodal information at a given point, including demographics, genetic information (e.g., the ApoE genes), cognitive test scores and brain MRIs. The method learns health status transitions, as well as how to map the health status abstraction to a diagnosis. An important innovation is the incorporation of an image extraction component in an end-to-end manner in the framework using hybrid convolutional layers, visual attention guided by domain knowledge and information theoretical measurements to extract different features from images. Moreover, we introduce methodology for the seamless transfer of the models between datasets collected as part of different studies, where the recorded information, including clinical tests, images collected and subject questionnaires, differs across study cohorts. The methods mitigate the challenges presented by this otherwise rich and varied data by using fused signals and mappings between abstractions. At the end of this study, we will have created a general forecasting framework, capable of predicting the onset of Alzheimer’s years before symptoms arise, a striking advance that will enable clinicians to identify new prevention strategies and prepare for, rather that respond to, Alzheimer’s.