Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition.
Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition.
复制标题
面部表情和脑电图融合用于多模态情绪识别
DOI：
10.1155/2017/2107451
复制
发表时间：
2017
影响因子：
--
通讯作者：
Pan J
中科院分区：
文献类型：
--
作者：
Huang Y;Yang J;Liao P;Pan J
本文提出了两种用于情感识别的大脑和外周信号多模态融合方法。输入信号是脑电信号和面部表情。刺激是基于电影片段的子集，对应于四个特定的区域的valance-arousal情绪空间（幸福，中性，悲伤和恐惧）。对于面部表情检测，通过神经网络分类器检测四种基本情感状态（快乐、中性、悲伤和恐惧）。对于脑电检测，四个基本的情绪状态和三个情绪强度水平（强，普通，和弱）检测两个支持向量机（SVM）分类器，分别。情感识别是基于两个决策层融合方法的EEG和面部表情检测通过使用求和规则或产生式规则。20名健康受试者参加了两个实验。结果表明，两种多模态融合检测的准确率分别为81.25%和82.75%，均高于表情检测（74.38%）和脑电检测（66.88%）。将表情信息和脑电信息结合起来进行情感识别，弥补了表情信息和脑电信息单一的缺陷。
This paper proposes two multimodal fusion methods between brain and peripheral signals for emotion recognition. The input signals are electroencephalogram and facial expression. The stimuli are based on a subset of movie clips that correspond to four specific areas of valance-arousal emotional space (happiness, neutral, sadness, and fear). For facial expression detection, four basic emotion states (happiness, neutral, sadness, and fear) are detected by a neural network classifier. For EEG detection, four basic emotion states and three emotion intensity levels (strong, ordinary, and weak) are detected by two support vector machines (SVM) classifiers, respectively. Emotion recognition is based on two decision-level fusion methods of both EEG and facial expression detections by using a sum rule or a production rule. Twenty healthy subjects attended two experiments. The results show that the accuracies of two multimodal fusion detections are 81.25% and 82.75%, respectively, which are both higher than that of facial expression (74.38%) or EEG detection (66.88%). The combination of facial expressions and EEG information for emotion recognition compensates for their defects as single information sources.
登录
查看更多内容
多模态 BCI：意识障碍患者的目标检测、多维控制和意识评估
影响因子：
20.6
作者：
Li, Yuanqing;Pan, Jiahui;Wu, Wei
通讯作者：
Wu, Wei
影响因子：
1.9
作者：
Gratch, J;Marsella, S
通讯作者：
Marsella, S
DOI：
10.1109/tnnls.2014.2342533
发表时间：
2015-07-01
影响因子：
10.4
作者：
Gu, Bin;Sheng, Victor S.;Li, Shuo
通讯作者：
Li, Shuo
影响因子：
5.7
作者：
Christensen, James C.;Estepp, Justin R.;Russell, Christopher A.
通讯作者：
Russell, Christopher A.
DOI：
10.1109/tsmcb.2004.825930
发表时间：
2004-06-01
影响因子：
--
作者：
Ma, L;Khorasani, K
通讯作者：
Khorasani, K