Region of Interest Based Graph Convolution: A Heatmap Regression Approach for Action Unit Detection
Region of Interest Based Graph Convolution: A Heatmap Regression Approach for Action Unit Detection
复制标题
DOI：
10.1145/3394171.3413674
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Zheng Zhang;Taoyue Wang;L. Yin
中科院分区：
文献类型：
--
作者：
Zheng Zhang;Taoyue Wang;L. Yin
人类面部表情的机器视觉已经研究了几十年，从原型表情到动作单元（AU），从手工制作到深层特征，从多类到多标签分类。由于广泛采用的深度网络缺乏对学习表示的解释，因此无法有效地强加和检查人类先验知识。另一方面，AU是人类定义的概念。为了符合这一想法，需要更精细的网络设计。在本文中，我们首先将热图扩展到 ROI 地图，对出现的 AU 的正负位置进行编码，然后采用精心设计的主干网络对其进行回归。这样，AU检测分两个阶段进行：关键区域定位和出现分类。为了提示 ROI 之间的空间依赖性，我们利用图卷积进行特征细化。相似度矩阵的分解由AU标签监督。这个新颖的框架在两个用于 AU 检测的基准数据库（BP4D 和 DISFA）上进行了评估。实验结果优于最先进的算法和基线模型，证明了我们提出的方法的有效性。
Machine vision of human facial expressions has been studied for decades, from prototypical expressions to Action Units (AUs), from hand-crafted to deep features, from multi-class to multi-label classifications. Since the widely adopted deep networks lack interpretation on learnt representations, human prior knowledge cannot be effectively imposed and examined. On the other hand, AU is a human defined concept. In order to align with this idea, a finer level of network design is desired. In this paper, we first extend the heatmaps to ROI maps, encoding the location of both positive and negative occurred AUs, then employ a well-designed backbone network to regress it. In this way, AU detection is performed in two stages, key regions localization and occurrence classification. To prompt the spatial dependency among ROIs, we utilize graph convolution for feature refinement. The decomposition of similarity matrix is supervised by AU labels. This novel framework is evaluated on two benchmark databases (BP4D and DISFA) for AU detection. The experimental results are superior to the state-of-the-art algorithms and baseline models, demonstrating the effectiveness of our proposed method.