Deep saliency models learn low-, mid-, and high-level features to predict scene attention.
Deep saliency models learn low-, mid-, and high-level features to predict scene attention.
复制标题
DOI：
10.1038/s41598-021-97879-z
复制
发表时间：
2021-09-16
影响因子：
4.6
通讯作者：
Henderson JM
中科院分区：
文献类型：
--
作者：
Hayes TR;Henderson JM
深度显着性模型代表了当前预测人类在现实世界场景中的位置的最新技术。然而，为了让深度显着性模型为注意力的认知理论提供信息，我们需要知道深度显着性模型如何优先考虑不同的场景特征来预测人们的目光。在这里，我们使用一种方法打开三个突出的深度显着性模型（MSI-Net，DeepGaze II和SAM-ResNet）的黑盒子，该方法对注意力，深度显着性模型输出以及低，中，高级别场景特征之间的关联进行建模。具体来说，我们通过将混合效应建模方法应用于大型眼动数据集，测量了每个深度显着性模型与低级别图像显着性，中级轮廓对称性和连接点以及高级意义之间的关联。我们发现，所有三个深度显着性模型与高层次和低层次的功能最密切相关，但表现出不同的功能权重和相互作用模式。这些发现表明，突出的深度显着性模型主要是学习与高级场景意义和低级图像显着性相关的图像特征，并强调了超越简单基准性能的重要性。
Deep saliency models represent the current state-of-the-art for predicting where humans look in real-world scenes. However, for deep saliency models to inform cognitive theories of attention, we need to know how deep saliency models prioritize different scene features to predict where people look. Here we open the black box of three prominent deep saliency models (MSI-Net, DeepGaze II, and SAM-ResNet) using an approach that models the association between attention, deep saliency model output, and low-, mid-, and high-level scene features. Specifically, we measured the association between each deep saliency model and low-level image saliency, mid-level contour symmetry and junctions, and high-level meaning by applying a mixed effects modeling approach to a large eye movement dataset. We found that all three deep saliency models were most strongly associated with high-level and low-level features, but exhibited qualitatively different feature weightings and interaction patterns. These findings suggest that prominent deep saliency models are primarily learning image features associated with high-level scene meaning and low-level image saliency and highlight the importance of moving beyond simply benchmarking performance.
登录
查看更多内容
DOI：
10.1037/0096-1523.25.1.210
发表时间：
1999-02-01
影响因子：
2.1
作者：
Henderson, JM;Weeks, PA;Hollingworth, A
通讯作者：
Hollingworth, A
影响因子：
2
作者：
Brockmole, James R.;Henderson, John M.
通讯作者：
Henderson, John M.
DOI：
10.1109/tpami.2017.2699184
发表时间：
2018-04-01
影响因子：
23.6
作者：
Chen, Liang-Chieh;Papandreou, George;Yuille, Alan L.
通讯作者：
Yuille, Alan L.
影响因子：
1.7
作者：
Hayes, Taylor R.;Henderson, John M.
通讯作者：
Henderson, John M.
影响因子：
29.9
作者：
Henderson, John M.;Hayes, Taylor R.
通讯作者：
Hayes, Taylor R.