Micro-Expression Recognition Base on Optical Flow Features and Improved MobileNetV2
Micro-Expression Recognition Base on Optical Flow Features and Improved MobileNetV2
复制标题
DOI：
10.3837/tiis.2021.06.002
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Wei Xu;Hao Zheng;Zhongxue Yang;Yingjie Yang
中科院分区：
文献类型：
--
作者：
Wei Xu;Hao Zheng;Zhongxue Yang;Yingjie Yang
当一个人试图隐藏情绪时，真正的情绪会以微表情的形式表现出来。人脸微表情识别的研究在模式识别领域仍极具挑战性。这是因为很难实现最好的特征提取方法来处理变化小、持续时间短的微表情。大多数方法都是基于手工制作的特征来提取微妙的面部动作。在这项研究中，我们介绍了一种融合了光流和深度学习的方法。首先，从每个视频序列中取出起始帧和顶点帧。然后，利用光流法提取这两帧之间的运动特征。最后，将特征输入到改进的MobileNetV2模型中，应用支持向量机对表情进行分类。为了评估该方法的有效性，我们在公共自然微表情数据库CASME II上进行了实验，在应用留一主题交叉验证方法的情况下，识别正确率达到了53.01%，F-Score达到了0.5231。实验结果表明，该方法能够显著提高微表情的识别性能。
When a person tries to conceal emotions, real emotions will manifest themselves in the form of micro-expressions. Research on facial micro-expression recognition is still extremely challenging in the field of pattern recognition. This is because it is difficult to implement the best feature extraction method to cope with micro-expressions with small changes and short duration. Most methods are based on hand-crafted features to extract subtle facial movements. In this study, we introduce a method that incorporates optical flow and deep learning. First, we take out the onset frame and the apex frame from each video sequence. Then, the motion features between these two frames are extracted using the optical flow method. Finally, the features are inputted into an improved MobileNetV2 model, where SVM is applied to classify expressions. In order to evaluate the effectiveness of the method, we conduct experiments on the public spontaneous micro-expression database CASME II. Under the condition of applying the leave-one-subject-out cross-validation method, the recognition accuracy rate reaches 53.01%, and the F-score reaches 0.5231. The results show that the proposed method can significantly improve the micro-expression recognition performance.