Multimodal learning for facial expression recognition
Multimodal learning for facial expression recognition
复制标题
DOI：
10.1016/j.patcog.2015.04.012
复制
发表时间：
2015-10
期刊：
影响因子：
--
通讯作者：
Wei Zhang;Youmei Zhang;Lin Ma;Jingwei Guan;S. Gong
中科院分区：
文献类型：
--
作者：
Wei Zhang;Youmei Zhang;Lin Ma;Jingwei Guan;S. Gong
本文提出了面部表情识别（FER）的多模态学习。多模态学习方法首次尝试通过考虑面部图像的纹理和标志模态来学习联合表示，这两者是互补的。为了学习每种模态的表示以及不同模态之间的相关性和相互作用，采用结构化正则化（SR）来分别强制和学习每种模态的特定于模态的稀疏性和密度。通过引入SR，充分考虑了面部表情的全面性，不仅可以处理细微的表情，而且对面部图像的不同输入具有鲁棒性。通过所提出的多模态学习网络，多模态输入的联合表示学习将更适合 FER。 CK+和NVIE数据库上的实验结果证明了我们提出的方法的优越性。
In this paper, multimodal learning for facial expression recognition (FER) is proposed. The multimodal learning method makes the first attempt to learn the joint representation by considering the texture and landmark modality of facial images, which are complementary with each other. In order to learn the representation of each modality and the correlation and interaction between different modalities, the structured regularization (SR) is employed to enforce and learn the modality-specific sparsity and density of each modality, respectively. By introducing SR, the comprehensiveness of the facial expression is fully taken into consideration, which can not only handle the subtle expression but also perform robustly to different input of facial images. With the proposed multimodal learning network, the joint representation learning from multimodal inputs will be more suitable for FER. Experimental results on the CK+ and NVIE databases demonstrate the superiority of our proposed method.