Hybrid Multimodality Fusion with Cross-Domain Knowledge Transfer to Forecast Progression Trajectories in Cognitive Decline.
Hybrid Multimodality Fusion with Cross-Domain Knowledge Transfer to Forecast Progression Trajectories in Cognitive Decline.
复制标题
混合多模态融合与跨领域知识转移来预测认知衰退的进展轨迹。
DOI：
10.1007/978-3-031-47425-5_24
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Liu,Mingxia
中科院分区：
文献类型：
--
作者：
Yu,Minhui;Liu,Yunbi;Wu,Jinjian;Bozoki,Andrea;Qiu,Shijun;Yue,Ling;Liu,Mingxia
磁共振成像（MRI）和正电子发射断层扫描（PET）越来越多地用于预测由临床前和前驱阿尔茨海默病（AD）引起的认知下降的进展轨迹。许多现有的研究已经探索了这两种不同模式与不同机器和深度学习方法的潜力。但是，成功融合MRI和PET可能是复杂的，因为它们具有独特的特征和缺失的模式。为此，我们开发了一个混合多模态融合（HMF）框架，该框架具有跨领域知识转移，用于联合MRI和PET表示学习，特征融合和认知衰退进展预测。我们的HMF由三个模块组成：1）一个模块，以填补丢失的PET图像，2）一个模块，以提取多模态特征的MRI和PET图像，和3）一个模块，以融合提取的多模态特征。为了解决小样本量的问题，我们采用了一种跨领域的知识转移策略，从ADNI数据集，其中包括795名受试者，到独立的小规模AD相关队列，以利用ADNI中存在的丰富知识。在三项AD相关研究中对拟定的HMF进行了广泛评估，其中包括272名跨多个疾病阶段的受试者，如主观认知下降和轻度认知障碍。实验结果表明，我们的方法优于几个国家的最先进的方法在预测AD相关的认知下降的进展轨迹。
Magnetic resonance imaging (MRI) and positron emission tomography (PET) are increasingly used to forecast progression trajectories of cognitive decline caused by preclinical and prodromal Alzheimer’s disease (AD). Many existing studies have explored the potential of these two distinct modalities with diverse machine and deep learning approaches. But successfully fusing MRI and PET can be complex due to their unique characteristics and missing modalities. To this end, we develop a hybrid multimodality fusion (HMF) framework with cross-domain knowledge transfer for joint MRI and PET representation learning, feature fusion, and cognitive decline progression forecasting. Our HMF consists of three modules: 1) a module to impute missing PET images, 2) a module to extract multimodality features from MRI and PET images, and 3) a module to fuse the extracted multimodality features. To address the issue of small sample sizes, we employ a cross-domain knowledge transfer strategy from the ADNI dataset, which includes 795 subjects, to independent small-scale AD-related cohorts, in order to leverage the rich knowledge present within the ADNI. The proposed HMF is extensively evaluated in three AD-related studies with 272 subjects across multiple disease stages, such as subjective cognitive decline and mild cognitive impairment. Experimental results demonstrate the superiority of our method over several state-of-the-art approaches in forecasting progression trajectories of AD-related cognitive decline.