Impact of Affective Multimedia Content on the Electroencephalogram and Facial Expressions
Impact of Affective Multimedia Content on the Electroencephalogram and Facial Expressions
复制标题
情感多媒体内容对脑电图和面部表情的影响
DOI：
10.1038/s41598-019-52891-2
复制
发表时间：
2019
影响因子：
4.6
通讯作者：
Sejnowski, Terrence J.
中科院分区：
文献类型：
--
作者：
Siddharth, Siddharth;Jung, Tzyy-Ping;Sejnowski, Terrence J.
情感计算领域的大部分研究都集中在通过脑电图或面部表情来检测和分类人类的情绪。设计多媒体内容以唤起某种情感在很大程度上是由用户提供的手动评级驱动的。在这里，我们提出的见解从情感特征之间的相关性三种模式，即情感多媒体内容，脑电图和面部表情。有趣的是，低水平的视听特征，如电影片段中视频的对比度和同质性以及音频的音调，与面部表情和脑电图的变化最为相关。我们还检测了与人脸和大脑相关的区域（除了脑电图频带），这些区域最能代表情感反应。三种模式之间的计算建模表明，这些区域的特征与用户报告的情感标签之间存在高度相关性。最后，以脑电图和面部图像为输入的卷积神经网络的不同层之间的相关性提供了对人类情感的见解。总之，这些发现将有助于(1)设计更有效的多媒体内容来吸引或影响观众，(2)理解情感的大脑/身体生物标记，以及(3)开发更新的脑机接口以及基于面部表情的算法来读取观众的情绪反应。
Most of the research in the field of affective computing has focused on detecting and classifying human emotions through electroencephalogram (EEG) or facial expressions. Designing multimedia content to evoke certain emotions has been largely motivated by manual rating provided by users. Here we present insights from the correlation of affective features between three modalities namely, affective multimedia content, EEG, and facial expressions. Interestingly, low-level Audio-visual features such as contrast and homogeneity of the video and tone of the audio in the movie clips are most correlated with changes in facial expressions and EEG. We also detect the regions associated with the human face and the brain (in addition to the EEG frequency bands) that are most representative of affective responses. The computational modeling between the three modalities showed a high correlation between features from these regions and user-reported affective labels. Finally, the correlation between different layers of convolutional neural networks with EEG and Face images as input provides insights into human affection. Together, these findings will assist in (1) designing more effective multimedia contents to engage or influence the viewers, (2) understanding the brain/body bio-markers of affection, and (3) developing newer brain-computer interfaces as well as facial-expression-based algorithms to read emotional responses of the viewers.
影响因子：
2.4
作者：
P. McCullagh;J. Nelder
通讯作者：
P. McCullagh;J. Nelder
音乐和感受的情绪：系统的音调水平变化如何影响愉悦和唤醒的体验
DOI：
--
发表时间：
2014
期刊：
影响因子：
--
作者：
Lucas Jaquet;B. Danuser;P. Gomez
通讯作者：
P. Gomez
新皮质伽马波与意识有关吗？
影响因子：
2.9
作者：
C. H. Vanderwolf
通讯作者：
C. H. Vanderwolf