Slices of Attention in Asynchronous Video Job Interviews
Slices of Attention in Asynchronous Video Job Interviews
复制标题
DOI：
10.1109/acii.2019.8925439
复制
发表时间：
2019-09
期刊：
影响因子：
--
通讯作者：
Léo Hemamou;G. Felhi;Jean-Claude Martin;C. Clavel
中科院分区：
文献类型：
--
作者：
Léo Hemamou;G. Felhi;Jean-Claude Martin;C. Clavel
非语言行为在招聘决策中的影响仍是一个悬而未决的问题。研究这个问题很重要，因为它可以更好地了解如何为求职面试培训应聘者，并让招聘人员意识到有影响力的非语言行为。最近，由于用于自动分析社会信号的工具的开发(面部表情检测、语音处理等)以及机器学习方法的出现，这项研究得到了加速。然而，这些研究仍然主要基于人工设计的特征，这限制了对有影响力的社会信号的发现。另一方面，深度学习方法是一种在不需要特征工程的情况下发现复杂模式的很有前途的工具。在本文中，我们重点研究了深度学习方法发现的异步工作视频面试中有影响力的非语言社交信号。我们使用以前发布的深度学习系统，该系统旨在推断应聘者关于一系列面试问题的受雇能力。这个系统的一个特殊性是使用了注意力机制，旨在识别答案的相关部分。因此，在细粒度的时间级别上的信息可以使用全局(在访谈级别)关于可Hiraiability的注释来提取。虽然大多数深度学习系统使用注意力机制来在注意力上升时提供切片的快速可视化，但我们进行了深入的分析，以了解在这些时刻发生了什么。首先，我们提出了一种自动提取注意力上升的切片(注意力切片)的方法。其次，我们通过与随机抽样的注意切片进行比较来研究注意切片的内容。最后，我们发现，与随机抽样的切片相比，它们承载了显著更多的关于可采性的信息，并且这些信息与与焦虑和话轮转换相关的视觉线索有关。
The impact of non verbal behaviour in a hiring decision remains an open question. Investigating this question is important, as it could provide a better understanding on how to train candidates for job interviews and make recruiters be aware of influential non verbal behaviour. This research has recently been accelerated due to the development of tools for the automatic analysis of social signals (facial expression detection, speech processing, etc), and the emergence of machine learning methods. However, these studies are still mainly based on hand engineered features, which imposes a limit to the discovery of influential social signals. On the other side, deep learning methods are a promising tool to discover complex patterns without the necessity of feature engineering. In this paper, we focus on studying influential non verbal social signals in asynchronous job video interviews that are discovered by deep learning methods. We use a previously published deep learning system that aims at inferring the hirability of a candidate with regard to a sequence of interview questions. One particularity of this system is the use of attention mechanisms, which aim at identifying the relevant parts of an answer. Thus, information at a fine-grained temporal level could be extracted using global (at the interview level) annotations on hirability. While most of the deep learning systems use attention mechanisms to offer a quick visualization of slices when a rise of attention occurs, we perform an in-depth analysis to understand what happens during these moments. First, we propose a methodology to automatically extract slices where there is a rise of attention (attention slices). Second, we study the content of attention slices by comparing them with randomly sampled slices. Finally, we show that they bear significantly more information for hirability than randomly sampled slices, and that such information is related to visual cues associated with anxiety and turn taking.