Uncertainty-Aware Gaze Tracking for Assisted Living Environments
Uncertainty-Aware Gaze Tracking for Assisted Living Environments
复制标题
DOI：
10.1109/tip.2023.3253253
复制
发表时间：
2023-04
影响因子：
10.6
通讯作者：
Paris Her;Logan Manderle;P. Dias;Henry Medeiros;F. Odone
中科院分区：
文献类型：
--
作者：
Paris Her;Logan Manderle;P. Dias;Henry Medeiros;F. Odone
有效的辅助生活环境必须能够推断出居住者在各种场景中如何互动。凝视方向提供了一个人如何与环境及其居住者互动的强烈指示。在本文中，我们研究了多摄像机辅助生活环境中的视线跟踪问题。我们提出了一种基于神经网络回归器生成的预测的视线跟踪方法，该方法仅依赖于面部关键点的相对位置来估计视线。对于每个凝视预测，我们的回归器还提供了自己的不确定性的估计，这是用来衡量以前估计的凝视在跟踪框架的基础上的角度卡尔曼滤波器的贡献。我们的凝视估计神经网络使用置信度门控单元来减轻涉及受试者的部分遮挡或不利视图的场景中的关键点预测不确定性。我们使用MoDiPro数据集的视频来评估我们的方法，该数据集是我们在真实的辅助生活设施中获得的，并在公开的MPIIFaceGaze，GazeFollow和Gaze360数据集上获得。实验结果表明，我们的凝视估计网络优于先进的方法，同时还提供了与相应估计的实际角度误差高度相关的不确定性预测。最后，我们的方法的时间整合性能的分析表明，它产生准确和时间稳定的凝视预测。
Effective assisted living environments must be able to infer how their occupants interact in a variety of scenarios. Gaze direction provides strong indications of how a person engages with the environment and its occupants. In this paper, we investigate the problem of gaze tracking in multi-camera assisted living environments. We propose a gaze tracking method based on predictions generated by a neural network regressor that relies only on the relative positions of facial keypoints to estimate gaze. For each gaze prediction, our regressor also provides an estimate of its own uncertainty, which is used to weigh the contribution of previously estimated gazes within a tracking framework based on an angular Kalman filter. Our gaze estimation neural network uses confidence gated units to alleviate keypoint prediction uncertainties in scenarios involving partial occlusions or unfavorable views of the subjects. We evaluate our method using videos from the MoDiPro dataset, which we acquired in a real assisted living facility, and on the publicly available MPIIFaceGaze, GazeFollow, and Gaze360 datasets. Experimental results show that our gaze estimation network outperforms sophisticated state-of-the-art methods, while additionally providing uncertainty predictions that are highly correlated with the actual angular error of the corresponding estimates. Finally, an analysis of the temporal integration performance of our method demonstrates that it generates accurate and temporally stable gaze predictions.