An Efficient Approach to Informative Feature Extraction from Multimodal Data
An Efficient Approach to Informative Feature Extraction from Multimodal Data
复制标题
DOI：
10.1609/aaai.v33i01.33015281
复制
发表时间：
2018-11
期刊：
影响因子：
--
通讯作者：
Lichen Wang;Jiaxiang Wu;Shao-Lun Huang;Lizhong Zheng;Xiangxiang Xu;Lin Zhang;Junzhou Huang
中科院分区：
文献类型：
--
作者：
Lichen Wang;Jiaxiang Wu;Shao-Lun Huang;Lizhong Zheng;Xiangxiang Xu;Lin Zhang;Junzhou Huang
多模态特征提取的一个主要焦点是找到最大相关的各个模态的表示。作为一种著名的相关性度量，HGR最大相关性是一个有吸引力的目标，因为它的操作意义和理想的性质。然而，严格的白化约束形式化的HGR最大相关限制了它的应用。为了解决这个问题，本文提出了Soft-HGR，一种新的框架，从多个数据模态提取信息特征。具体来说，我们的框架防止了“硬”白化约束，同时保留了与HGR最大相关性相同的特征几何。Soft-HGR的优化目标简单，只涉及两个内积，保证了优化的效率和稳定性。我们进一步推广的框架，以处理两个以上的模态和缺失的模态。当标签部分可用时，我们通过进行半监督自适应来增强特征表示的区分能力。经验评估表明，我们的方法学习更多的信息特征映射，是更有效的优化。
One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation be-´ comes an appealing objective because of its operational meaning and desirable properties. However, the strict whitening constraints formalized in the HGR maximal correlation limit its application. To address this problem, this paper proposes Soft-HGR, a novel framework to extract informative features from multiple data modalities. Specifically, our framework prevents the “hard” whitening constraints, while simultaneously preserving the same feature geometry as in the HGR maximal correlation. The objective of Soft-HGR is straightforward, only involving two inner products, which guarantees the efficiency and stability in optimization. We further generalize the framework to handle more than two modalities and missing modalities. When labels are partially available, we enhance the discriminative power of the feature representations by making a semi-supervised adaptation. Empirical evaluation implies that our approach learns more informative feature mappings and is more efficient to optimize.