Detection of eye contact with deep neural networks is as accurate as human experts.
Detection of eye contact with deep neural networks is as accurate as human experts.
复制标题
DOI：
10.1038/s41467-020-19712-x
复制
发表时间：
2020-12-14
影响因子：
16.6
通讯作者：
Rehg JM
中科院分区：
文献类型：
--
作者：
Chong E;Clark-Whitney E;Southerland A;Stubbs E;Miller C;Ajodan EL;Silverman MR;Lord C;Rozga A;Jones RM;Rehg JM
眼神交流是人类最主要的社交方式之一。作为社会角色和沟通技能分析的一部分，以及对于临床筛查，眼神接触的量化非常有价值。估计主体的观看方向是一项具有挑战性的任务，但可以通过提供独特视角的可穿戴式视点相机有效地捕获目光接触。虽然从这个角度来看，目光接触的时刻可以手工编码，但这样的过程往往是费力和主观的。在这项工作中，我们开发了一个深度神经网络模型来自动检测以自我为中心的视频中的目光接触。它是第一个达到与人类专家相当的准确性的。我们使用4，339，879张注释图像的数据集训练深度卷积网络，其中包括103名具有不同人口背景的受试者。57名受试者被诊断为自闭症谱系障碍。该网络在18个验证对象上实现了0.936的整体精度和0.943的召回率，其性能与10个经过训练的人类编码器相当，平均精度为0.918，召回率为0.946。我们的方法将有助于凝视行为分析，作为一个可扩展的，客观的，可访问的工具，为临床医生和研究人员。目光接触是一种重要的社会行为，它的测量可以帮助自闭症的诊断和治疗。在这里，作者表明，深度神经网络模型可以像人类专家一样准确地检测眼神交流。
Eye contact is among the most primary means of social communication used by humans. Quantification of eye contact is valuable as a part of the analysis of social roles and communication skills, and for clinical screening. Estimating a subject’s looking direction is a challenging task, but eye contact can be effectively captured by a wearable point-of-view camera which provides a unique viewpoint. While moments of eye contact from this viewpoint can be hand-coded, such a process tends to be laborious and subjective. In this work, we develop a deep neural network model to automatically detect eye contact in egocentric video. It is the first to achieve accuracy equivalent to that of human experts. We train a deep convolutional network using a dataset of 4,339,879 annotated images, consisting of 103 subjects with diverse demographic backgrounds. 57 subjects have a diagnosis of Autism Spectrum Disorder. The network achieves overall precision of 0.936 and recall of 0.943 on 18 validation subjects, and its performance is on par with 10 trained human coders with a mean precision 0.918 and recall 0.946. Our method will be instrumental in gaze behavior analysis by serving as a scalable, objective, and accessible tool for clinicians and researchers. Eye contact is a key social behavior and its measurement could facilitate the diagnosis and treatment of autism. Here the authors show that a deep neural network model can detect eye contact as accurately has human experts.
登录
查看更多内容
DOI：
10.1177/1362361318766247
发表时间：
2019-04
期刊：
Autism : the international journal of research and practice
影响因子：
--
作者：
Campbell K;Carpenter KL;Hashemi J;Espinosa S;Marsan S;Borg JS;Chang Z;Qiu Q;Vermeer S;Adler E;Tepper M;Egger HL;Baker JP;Sapiro G;Dawson G
通讯作者：
Dawson G
影响因子：
64.8
作者：
Esteva A;Kuprel B;Novoa RA;Ko J;Swetter SM;Blau HM;Thrun S
通讯作者：
Thrun S
影响因子：
3.9
作者：
Hus, Vanessa;Lord, Catherine
通讯作者：
Lord, Catherine
影响因子：
3.9
作者：
Grzadzinski, Rebecca;Carr, Themba;Lord, Catherine
通讯作者：
Lord, Catherine
DOI：
10.1073/pnas.0502205102
发表时间：
2005-11-22
影响因子：
11.1
作者：
Farroni, T;Johnson, MH;Csibra, G
通讯作者：
Csibra, G