Target-absent Human Attention
Target-absent Human Attention
复制标题
DOI：
10.48550/arxiv.2207.01166
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Zhibo Yang;Sounak Mondal;Seoyoung Ahn;G. Zelinsky;Minh Hoai;D. Samaras
中科院分区：
文献类型：
--
作者：
Zhibo Yang;Sounak Mondal;Seoyoung Ahn;G. Zelinsky;Minh Hoai;D. Samaras
人类注视行为的预测对于构建能够预测用户注意力的人机交互系统是重要的。计算机视觉模型已经被开发来预测人们在搜索目标物体时所做的注视。但如果目标不在图像中呢？同样重要的是要知道人们在找不到目标时如何搜索，以及何时会停止搜索。在本文中，我们提出了一个数据驱动的计算模型，解决了搜索终止问题，并预测搜索固定的人搜索不出现在图像中的目标的扫描路径。我们将视觉搜索建模为模仿学习问题，并使用我们称为Foveated Feature Maps（FFM）的新状态表示来表示观众通过固定获得的内部知识。FFM将模拟的中央凹视网膜集成到预训练的ConvNet中，该ConvNet产生网络内特征金字塔，所有这些都具有最小的计算开销。我们的方法集成了FFM作为逆强化学习中的状态表示。在实验上，我们改进了COCO-Search 18数据集上预测人类目标缺失搜索行为的最新技术。代码可从以下网址获得：https://github.com/cvlab-stonybrook/Target-absent-Human-Attention。
The prediction of human gaze behavior is important for building human-computer interaction systems that can anticipate the user's attention. Computer vision models have been developed to predict the fixations made by people as they search for target objects. But what about when the target is not in the image? Equally important is to know how people search when they cannot find a target, and when they would stop searching. In this paper, we propose a data-driven computational model that addresses the search-termination problem and predicts the scanpath of search fixations made by people searching for targets that do not appear in images. We model visual search as an imitation learning problem and represent the internal knowledge that the viewer acquires through fixations using a novel state representation that we call Foveated Feature Maps (FFMs). FFMs integrate a simulated foveated retina into a pretrained ConvNet that produces an in-network feature pyramid, all with minimal computational overhead. Our method integrates FFMs as the state representation in inverse reinforcement learning. Experimentally, we improve the state of the art in predicting human target-absent search behavior on the COCO-Search18 dataset. Code is available at: https://github.com/cvlab-stonybrook/Target-absent-Human-Attention.