Facial Expression Recognition With Deeply-Supervised Attention Network
Facial Expression Recognition With Deeply-Supervised Attention Network
复制标题
深度监督注意力网络的面部表情识别
DOI：
10.1109/taffc.2020.2988264
复制
发表时间：
2022-04-01
影响因子：
11.2
通讯作者：
Lam, Jacqueline C. K.
中科院分区：
文献类型：
--
作者：
Fan, Yingruo;Li, Victor O. K.;Lam, Jacqueline C. K.
面部表情识别（FER）是社交的关键。然而，目前的研究在解决由于人口统计学差异，如种族，性别和年龄等，面部表情差异时存在局限性。在这篇文章中，我们首先提出了一个深度监督的注意力网络（DSAN），以识别人类的情绪自动面部图像的基础上。基于DSAN，设计了一个两阶段的训练方案，充分利用了种族/性别/年龄的相关信息。在我们的DSAN框架中，利用多尺度特征来捕获从深层到浅层的更多区分信息。此外，我们采用的注意力块，突出必要的本地面部特征，它表现良好，当它被纳入深度监督框架。最后，我们以深度监督的方式将多个卷积层的互补特征联合收割机组合起来，并集成中间预测分数。我们的实验结果表明，我们提出的框架可以（i）有效地整合人口统计信息，提高各种FER任务的性能，（ii）通过捕获感兴趣区域（ROI），学习具有视觉解释的信息特征表示，（iii）对姿势和自发FER数据库都实现上级性能，每个数据库包含不同性别的人类面部表情的图片，年龄或种族。
Facial expression recognition (FER) is crucial for social communication. However, current studies present limitations when addressing facial expression difference due to demographic variation, such as race, gender, and age, etc. In this article, we first propose a deeply-supervised attention network (DSAN) to recognize human emotions based on facial images automatically. Based on DSAN, a two-stage training scheme is designed, taking full advantage of the race/gender/age-related information. In our DSAN framework, multi-scale features are leveraged to capture more discriminative information from the deep layers to the shallow layers. Furthermore, we adopt the attention block to highlight the essential local facial characteristics; it performs well when it is incorporated into the deeply-supervised framework. Finally, we combine the complementary characteristics of multiple convolutional layers in deeply-supervised manner and ensemble the intermediate predicted scores. Our experimental results have shown that our proposed framework can (i) effectively integrate demographic information in improving the performance of a variety of FER tasks, (ii) learn informative feature representations with a visual explanation by capturing the regions of interests (ROI), (iii) achieve superior performance for both the posed and the spontaneous FER databases, each containing pictures of human facial expressions varied in gender, age or race.