POSTER: A Pyramid Cross-Fusion Transformer Network for Facial Expression Recognition
POSTER: A Pyramid Cross-Fusion Transformer Network for Facial Expression Recognition
复制标题
DOI：
10.1109/iccvw60793.2023.00339
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Ce Zheng;Mat'ias Mendieta;Chen Chen-Chen
中科院分区：
文献类型：
--
作者：
Ce Zheng;Mat'ias Mendieta;Chen Chen-Chen
人脸表情识别（FER）是计算机视觉中的一项重要任务，在人机交互、教育、医疗保健和在线监控等领域具有实际应用。在这个具有挑战性的FER任务中，有三个关键问题特别普遍：类间相似性，类内差异和尺度敏感性。虽然现有的工作通常解决其中一些问题，但没有一个在统一的框架内充分解决所有三个挑战。在本文中，我们提出了一个双流金字塔交叉融合转换Transformer网络（POSTER），旨在全面解决这三个问题。具体来说，我们设计了一种基于变换的交叉融合方法，使面部标志性特征和图像特征的有效合作，以最大限度地适当注意到显着的面部区域。此外，POSTER采用金字塔结构，以促进规模不变性。大量的实验结果表明，我们的POSTER在RAF-DB（92.05%），FERPlus（91.62%）以及AffectNet 7类（67.31%）和8类（63.34%）上实现了新的最先进的结果。代码可在https://github.com/zczcwh/POSTER上获得。
Facial expression recognition (FER) is an important task in computer vision, having practical applications in areas such as human-computer interaction, education, health-care, and online monitoring. In this challenging FER task, there are three key issues especially prevalent: inter-class similarity, intra-class discrepancy, and scale sensitivity. While existing works typically address some of these issues, none have fully addressed all three challenges in a unified framework. In this paper, we propose a two-stream Pyramid crOss-fuSion TransformER network (POSTER), that aims to holistically solve all three issues. Specifically, we design a transformer-based cross-fusion method that enables effective collaboration of facial landmark features and image features to maximize proper attention to salient facial regions. Furthermore, POSTER employs a pyramid structure to promote scale invariance. Extensive experimental results demonstrate that our POSTER achieves new state-of-the-art results on RAF-DB (92.05%), FERPlus (91.62%), as well as AffectNet 7 class (67.31%) and 8 class (63.34%). Code is available at https://github.com/zczcwh/POSTER.