Automatic 2.5-D Facial Landmarking and Emotion Annotation for Social Interaction Assistance
Automatic 2.5-D Facial Landmarking and Emotion Annotation for Social Interaction Assistance
复制标题
DOI：
10.1109/tcyb.2015.2461131
复制
发表时间：
2016-09
影响因子：
11.8
通讯作者：
Xi Zhao;Jianhua Zou;Huibin Li;E. Dellandréa;I. Kakadiaris;Liming Chen
中科院分区：
文献类型：
--
作者：
Xi Zhao;Jianhua Zou;Huibin Li;E. Dellandréa;I. Kakadiaris;Liming Chen
视力低下、阿尔茨海默病和自闭症谱系障碍的人在社交生活中感知或解释面部情感表达方面存在困难。尽管2-D视频上的自动面部表情识别(FER)方法已经被广泛研究，但它们的性能受到头部姿势和光照条件的挑战的限制。三维人脸数据中的形状信息可以减少甚至克服这些挑战。然而，3D相机的高昂成本阻碍了它们的广泛使用。幸运的是，来自新兴便携式RGB-D相机的2.5-D面部数据为这一困境提供了一个很好的平衡。本文提出了一种基于RGB-D摄像机采集的2.5维人脸数据的自动情感标注解决方案。该解决方案由人脸标记法和FER方法组成。具体地说，我们建议建立一个可变形的局部人脸模型，并将该模型拟合到一个2.5维的人脸上，用于自动定位人脸地标。在FER中，提出了一种新的基于行动单元(AU)的空间FER方法。使用地标提取面部特征，并进一步表示为AU空间中的坐标，这些坐标被分类为面部表情。在Eurecm、FRGC和Bsporus三个公开可访问的人脸数据库上进行评估，所提出的人脸标志点和表情识别方法取得了令人满意的结果。还讨论了使用我们的算法在现实世界中可能的应用。
People with low vision, Alzheimer's disease, and autism spectrum disorder experience difficulties in perceiving or interpreting facial expression of emotion in their social lives. Though automatic facial expression recognition (FER) methods on 2-D videos have been extensively investigated, their performance was constrained by challenges in head pose and lighting conditions. The shape information in 3-D facial data can reduce or even overcome these challenges. However, high expenses of 3-D cameras prevent their widespread use. Fortunately, 2.5-D facial data from emerging portable RGB-D cameras provide a good balance for this dilemma. In this paper, we propose an automatic emotion annotation solution on 2.5-D facial data collected from RGB-D cameras. The solution consists of a facial landmarking method and a FER method. Specifically, we propose building a deformable partial face model and fit the model to a 2.5-D face for localizing facial landmarks automatically. In FER, a novel action unit (AU) space-based FER method has been proposed. Facial features are extracted using landmarks and further represented as coordinates in the AU space, which are classified into facial expressions. Evaluated on three publicly accessible facial databases, namely EURECOM, FRGC, and Bosphorus databases, the proposed facial landmarking and expression recognition methods have achieved satisfactory results. Possible real-world applications using our algorithms have also been discussed.