Dynamic 3D Gaze from Afar: Deep Gaze Estimation from Temporal Eye-Head-Body Coordination
Dynamic 3D Gaze from Afar: Deep Gaze Estimation from Temporal Eye-Head-Body Coordination
复制标题
DOI：
10.1109/cvpr52688.2022.00223
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Soma Nonaka;S. Nobuhara;K. Nishino
中科院分区：
文献类型：
--
作者：
Soma Nonaka;S. Nobuhara;K. Nishino
我们介绍了一种新的方法和数据集的3D凝视估计的自由移动的人从远处，通常在监视视图。在这种情况下，由于遮挡和缺乏分辨率，眼睛无法清楚地看到。现有的凝视估计方法遭受或退回到近似具有头部姿势的凝视，因为它们主要依赖于眼睛的清晰的特写视图。我们的主要想法是利用人们内在的目光，头部和身体协调。我们的方法将凝视估计公式化为贝叶斯预测，给出头部和身体方向的时间估计，可以从远处可靠地估计。我们建模的头部和身体的方向似然性和条件先验的注视方向上的那些与单独的神经网络，然后级联输出的3D注视方向。我们引入了一个广泛的新数据集，该数据集由5个室内和室外场景中捕获的带有3D凝视方向注释的监控视频组成。在此数据集和其他数据集上的实验结果验证了我们方法的准确性，并表明即使在摄像头看不到人脸的情况下，也可以从典型的监视距离准确地估计视线。
We introduce a novel method and dataset for 3D gaze estimation of a freely moving person from a distance, typically in surveillance views. Eyes cannot be clearly seen in such cases due to occlusion and lacking resolution. Existing gaze estimation methods suffer or fall back to approximating gaze with head pose as they primarily rely on clear, close-up views of the eyes. Our key idea is to instead leverage the intrinsic gaze, head, and body coordination of people. Our method formulates gaze estimation as Bayesian prediction given temporal estimates of head and body orientations which can be reliably estimated from a far. We model the head and body orientation likelihoods and the conditional prior of gaze direction on those with separate neural networks which are then cascaded to output the 3D gaze direction. We introduce an extensive new dataset that consists of surveillance videos annotated with 3D gaze directions captured in 5 indoor and outdoor scenes. Experimental results on this and other datasets validate the accuracy of our method and demonstrate that gaze can be accurately estimated from a typical surveillance distance even when the person's face is not visible to the camera.