Analyzing the Effect of Diverse Gaze and Head Direction on Facial Expression Recognition With Photo-Reflective Sensors Embedded in a Head-Mounted Display
Analyzing the Effect of Diverse Gaze and Head Direction on Facial Expression Recognition With Photo-Reflective Sensors Embedded in a Head-Mounted Display
复制标题
DOI：
10.1109/tvcg.2022.3179766
复制
发表时间：
2022-06
影响因子：
5.2
通讯作者：
Fumihiko Nakamura;Masaaki Murakami;Katsuhiro Suzuki;M. Fukuoka;Katsutoshi Masai;M. Sugimoto
中科院分区：
文献类型：
--
作者：
Fumihiko Nakamura;Masaaki Murakami;Katsuhiro Suzuki;M. Fukuoka;Katsutoshi Masai;M. Sugimoto
作为用于头戴式显示器（HMD）用户的面部表情识别技术之一，已经使用了嵌入式光反射传感器。在本文中，我们研究如何凝视和人脸方向影响面部表情识别使用嵌入式光反射传感器。首先，我们收集了五种面部表情的数据集（中性，快乐，愤怒，悲伤，惊讶），同时通过移动1）眼睛和2）头部来观察不同的方向。使用数据集，我们分析了视线和面部方向的影响，通过构建面部表情分类器的五种方式，并评估每个分类器的分类精度。结果显示，学习所有注视点数据的单个分类器实现了最高的分类性能。然后，我们调查了面部哪个部位受到注视和面部方向的影响。结果表明，注视方向影响上面部，而面部方向影响下面部。此外，通过消除面部表情再现性的偏差，我们研究了三种条件下的凝视和面部方向的纯效果。结果表明，在注视方向方面，为每个方向构建分类器显著提高了分类精度。然而，在面部方向方面，分类器条件之间存在轻微差异。我们的实验结果表明，多个分类器对应于多个凝视和人脸方向提高面部表情识别的准确性，但收集的数据的垂直运动的凝视和人脸是一个实用的解决方案，以提高面部表情识别的准确性。
As one of the facial expression recognition techniques for Head-Mounted Display (HMD) users, embedded photo-reflective sensors have been used. In this paper, we investigate how gaze and face directions affect facial expression recognition using the embedded photo-reflective sensors. First, we collected a dataset of five facial expressions (Neutral, Happy, Angry, Sad, Surprised) while looking in diverse directions by moving 1) the eyes and 2) the head. Using the dataset, we analyzed the effect of gaze and face directions by constructing facial expression classifiers in five ways and evaluating the classification accuracy of each classifier. The results revealed that the single classifier that learned the data for all gaze points achieved the highest classification performance. Then, we investigated which facial part was affected by the gaze and face direction. The results showed that the gaze directions affected the upper facial parts, while the face directions affected the lower facial parts. In addition, by removing the bias of facial expression reproducibility, we investigated the pure effect of gaze and face directions in three conditions. The results showed that, in terms of gaze direction, building classifiers for each direction significantly improved the classification accuracy. However, in terms of face directions, there were slight differences between the classifier conditions. Our experimental results implied that multiple classifiers corresponding to multiple gaze and face directions improved facial expression recognition accuracy, but collecting the data of the vertical movement of gaze and face is a practical solution to improving facial expression recognition accuracy.