Patch-level Gaze Distribution Prediction for Gaze Following
Patch-level Gaze Distribution Prediction for Gaze Following
复制标题
DOI：
10.1109/wacv56688.2023.00094
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Qiaomu Miao;Minh Hoai;D. Samaras
中科院分区：
文献类型：
--
作者：
Qiaomu Miao;Minh Hoai;D. Samaras
凝视跟踪的目的是通过预测目标位置，或表明目标位于图像之外，来预测一个人在场景中所看的位置。最近的研究通过训练具有逐像素均方误差（MSE）损失的热图回归任务来检测注视目标，同时将输入/输出预测任务制定为二值分类任务。这种训练公式对训练中可用的单个注释以更高的分辨率施加了严格的像素级约束，并且不考虑注释方差和两个子任务之间的相关性。为了解决这些问题，我们引入了斑块分布预测（PDP）方法。我们用PDP分支取代了以前模型中的in/out预测分支，通过预测也考虑外部情况的补丁级凝视分布。实验表明，我们的模型通过在标注方差较大的图像上预测更好的热图分布来正则化MSE损失，同时弥合了目标预测和进出预测子任务之间的差距，在公共注视跟踪数据集上，这两个子任务的性能都有显著提高。
Gaze following aims to predict where a person is looking in a scene, by predicting the target location, or indicating that the target is located outside the image. Recent works detect the gaze target by training a heatmap regression task with a pixel-wise mean-square error (MSE) loss, while formulating the in/out prediction task as a binary classification task. This training formulation puts a strict, pixel-level constraint in higher resolution on the single annotation available in training, and does not consider annotation variance and the correlation between the two subtasks. To address these issues, we introduce the patch distribution prediction (PDP) method. We replace the in/out prediction branch in previous models with the PDP branch, by predicting a patch-level gaze distribution that also considers the outside cases. Experiments show that our model regularizes the MSE loss by predicting better heatmap distributions on images with larger annotation variances, meanwhile bridging the gap between the target prediction and in/out prediction subtasks, showing a significant improvement in performance on both subtasks on public gaze following datasets.