Early Alzheimers Forecasting from Multimodal Data via Deep Transfer Learning, Evaluated on a Large-Scale Prospective Cohort Study

项目摘要 阿尔茨海默病是一种无法治愈的衰弱和退化性脑部疾病，影响约580万人 在美国的人。该项目将开发培训、调整和转移模型的技术， 从多模态数据中早期检测阿尔茨海默病，包括遗传信息，大脑 MRI和认知测试，重点是在一般人群中筛查AD（即，一次评估 横断面、前瞻性队列研究，代表人群）。 我们将介绍基于深度迁移学习的新技术，从大脑中提取表征 MRI，适用于前瞻性收集的数据，不附带专家注释。我们将 使用多模态深度学习将特征提取合并到端到端预测框架中。 这些方法将有助于建模，监测和预测阿尔茨海默氏症的进展 疾病，其中MRI伴随着以不同粒度级别收集的临床信息。我们 将从一个预测AD演变的模型开始，该模型是在来自 阿尔茨海默病神经影像学倡议（ADNI）研究。在ADNI数据上训练的模型通常依赖于 大脑核磁共振成像的专业工程特征需要相当大的领域， 知识和预处理，并且如果患者要获得 在医院做核磁共振扫描。因此，我们训练基于CNN的模型，这些模型直接与大脑MRI一起工作， 优化以捕获ADNI中存在的工程特性的预测能力。我们整合 大脑MRI网络，具有使用深度学习提取抽象表示的预测模型 根据受试者在给定时间点的多模态信息，评估受试者的健康状况，包括 人口统计学，遗传信息（例如，ApoE基因）、认知测试分数和脑MRI。的 方法学习健康状态转换，以及如何将健康状态抽象映射到诊断。 一个重要的创新是以端到端的方式纳入图像提取组件 在使用混合卷积层的框架中，由领域知识引导的视觉注意力， 信息理论测量，以从图像中提取不同的特征。 此外，我们还介绍了在收集的数据集之间无缝传输模型的方法 作为不同研究的一部分，其中记录的信息，包括临床测试，收集的图像和 受试者问卷，在研究队列中不同。这些方法减轻了由此带来的挑战。 通过使用融合的信号和抽象之间的映射，可以获得丰富多样的数据。 在这项研究的最后，我们将创建一个通用的预测框架，能够预测 在症状出现前几年就开始出现阿尔茨海默氏症，这一惊人的进步将使临床医生能够 确定新的预防策略，为阿尔茨海默氏症做好准备，而不是对它做出反应。

Project Summary Alzheimer's Disease, a debilitating and degenerative brain disease that has no cure, affects ~5.8 million people in the United States. This project will develop techniques to train, adapt and transfer models for the early detection of Alzheimer’s disease from multimodal data, including genetic information, brain MRIs and cognitive tests, with a focus on screening for AD in the general population (i.e., evaluated on a cross-sectional, prospective cohort study, representative of the populations). We will introduce new techniques, based on deep transfer learning, to extract representations from brain MRIs, applicable to prospectively collected data which is unaccompanied by expert annotations. We will incorporate the feature extraction in an end-to-end predictive framework using multimodal deep learning. Such methods will be useful for modeling, monitoring, and forecasting the progression of Alzheimer's disease, where MRIs accompany the clinical information collected at different levels of granularity. We will start with a model that predicts the evolution of AD, trained on multimodal longitudinal data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) study. Models trained on ADNI data typically rely on specialized engineered features from the brain MRIs requiring a considerable amount of domain knowledge and pre-processing and which would not be generally available if a patient were to obtain an MRI scan in the hospital. Thus, we train CNN-based models that work directly with brain MRIs, optimized to capture the predictive capabilities of the engineered features present in ADNI. We integrate the brain MRI network with a forecasting model that uses deep learning to extract abstract representations of the subjects' health status based on their multimodal information at a given point, including demographics, genetic information (e.g., the ApoE genes), cognitive test scores and brain MRIs. The method learns health status transitions, as well as how to map the health status abstraction to a diagnosis. An important innovation is the incorporation of an image extraction component in an end-to-end manner in the framework using hybrid convolutional layers, visual attention guided by domain knowledge and information theoretical measurements to extract different features from images. Moreover, we introduce methodology for the seamless transfer of the models between datasets collected as part of different studies, where the recorded information, including clinical tests, images collected and subject questionnaires, differs across study cohorts. The methods mitigate the challenges presented by this otherwise rich and varied data by using fused signals and mappings between abstractions. At the end of this study, we will have created a general forecasting framework, capable of predicting the onset of Alzheimer’s years before symptoms arise, a striking advance that will enable clinicians to identify new prevention strategies and prepare for, rather that respond to, Alzheimer’s.