Fusion of facial expressions and EEG for implicit affective tagging
Fusion of facial expressions and EEG for implicit affective tagging
复制标题
DOI：
10.1016/j.imavis.2012.10.002
复制
发表时间：
2013-02-01
影响因子：
4.7
通讯作者：
Patras, Ioannis
中科院分区：
文献类型：
--
作者：
Koelstra, Sander;Patras, Ioannis
近年来，用户生成的、未加标签的多媒体数据的爆炸性增长，产生了对这些数据的有效搜索和检索的强烈需求。基于内容的标记的主要方法是通过缓慢的、劳动密集型的手动注释。因此，自动标注目前是一个深入研究的课题。然而，很明显，在可预见的未来，这一过程不会完全自动化。我们建议让用户参与并研究隐式标记的方法，通过分析用户对与多媒体内容交互的反应来生成描述性标记。在此，我们提出了一种同时分析面部表情和脑电(EEG)信号的多模式方法来生成情感标记。我们在价唤醒空间中进行分类和回归，并给出了特征级和决策级融合的结果。当使用两种模式时，我们表现出结果的改善，这表明这两种模式包含互补的信息。(C)2012爱思唯尔B.V.保留所有权利。
The explosion of user-generated, untagged multimedia data in recent years, generates a strong need for efficient search and retrieval of this data. The predominant method for content-based tagging is through slow, labor-intensive manual annotation. Consequently, automatic tagging is currently a subject of intensive research. However, it is clear that the process will not be fully automated in the foreseeable future. We propose to involve the user and investigate methods for implicit tagging, wherein users' responses to the interaction with the multimedia content are analyzed in order to generate descriptive tags.Here, we present a multi-modal approach that analyses both facial expressions and electroencephalography (EEG) signals for the generation of affective tags. We perform classification and regression in the valence-arousal space and present results for both feature-level and decision-level fusion. We demonstrate improvement in the results when using both modalities, suggesting the modalities contain complementary information. (C) 2012 Elsevier B.V. All rights reserved.