Cross-modal Memory Fusion Network for Multimodal Sequential Learning with Missing Values
Cross-modal Memory Fusion Network for Multimodal Sequential Learning with Missing Values
复制标题
DOI：
10.1007/978-3-030-72240-1_30
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Chen Lin;Joyce C. Ho;Eugene Agichtein
中科院分区：
文献类型：
--
作者：
Chen Lin;Joyce C. Ho;Eugene Agichtein
许多现实应用中的信息本质上是多模式的、连续的，并且具有各种缺失值的特征。现有的插补方法主要关注一种模态的循环动态，而忽略了其他模态的互补特性。在本文中，我们提出了一种称为跨模态记忆融合网络（CMFN）的新方法，该方法显式学习特定模态和跨模态动态，以在多模态顺序学习任务中估算缺失值。对两个数据集的实验表明，我们的方法优于最先进的方法，并显示出其更好地估算复杂多模态数据集中缺失值的潜力。
Information in many real-world applications is inherently multi-modal, sequential and characterized by a variety of missing values. Existing imputation methods mainly focus on the recurrent dynamics in one modality while ignoring the complementary property from other modalities. In this paper, we propose a novel method called cross-modal memory fusion network (CMFN) that explicitly learns both modal-specific and cross-modal dynamics for imputing the missing values in multi-modal sequential learning tasks. Experiments on two datasets demonstrate that our method outperforms state-of-the-art methods and show its potential to better impute missing values in complex multi-modal datasets.