权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

RI: Medium: Inverse Reinforcement Learning for Human Attention Modeling

RI：媒介：人类注意力建模的逆强化学习

基本信息

批准号：
1763981
负责人：
Minh Hoai Nguyen
金额：
$ 119.9万
依托单位：
SUNY at Stony Brook
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-06-15 至 2023-05-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1763981&HistoricalAwards=false
关键词：
RI Medium Inverse Reinforcement Learning

项目摘要

The process by which people shift their attention from one thing to another touches upon everything that we think and do, and as such has widespread importance in fields ranging from basic research and education to applications in industry and national defense. This research develops a computational model for predicting these human shifts in visual attention. Prediction is understanding, and with this model we will achieve a greater understanding of this core human cognitive process. More tangibly, prediction enables applications to anticipate where attention will shift in response to seeing specific imagery. This in turn would usher in 1) a new generation of human-computer interactive systems, ones capable of interacting with users at the level of their attention movements, and 2) novel ways to annotate and index visual content based on attentional importance or interest. This project combines computational work with cognitive science and digital media, providing an entry to computer science through valuable learning experiences for women and underrepresented minorities who might otherwise be intimidated by traditional computational work. The project broadens the exposure of underrepresented minorities to STEM through partnerships with several ongoing efforts at Stony Brook University, including the Women in Science and Engineering (WISE) and Louis Stokes Alliance for Minority Participation (LSAMP) initiatives.This project investigates a synergistic computational and behavioral approach for modeling the movements of human attention. This approach is based on an assumption that attentional engagement on an image (or video frame) depends on both the pixels that are being viewed and the viewer's previous state. Based on this assumption, visual attention is posed as a Markov decision process, and inverse reinforcement learning is used to learn a reward function to associate specific spatio-temporal regions in an image, corresponding to the pixels at a viewer's momentary locus of attention, with a reward. Under this novel approach, the attention mechanism is treated as an agent whose action is to select a location in an image or image frame that will maximize its total reward. This model is being evaluated against a behavioral ground truth consisting of the eye movements that people make as they view images and video in the context of free viewing and visual search tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人们将注意力从一件事转移到另一件事的过程涉及到我们所想所做的一切，因此在从基础研究和教育到工业和国防应用的各个领域都具有广泛的重要性。这项研究开发了一个计算模型来预测这些人类视觉注意力的变化。预测就是理解，有了这个模型，我们将更好地理解这个核心的人类认知过程。更具体地说，预测使应用程序能够预测注意力在看到特定图像时会转移到哪里。这反过来又会带来1）新一代的人机交互系统，能够在用户的注意力运动水平上与用户交互，以及2）基于注意力重要性或兴趣来注释和索引视觉内容的新方法。该项目将计算工作与认知科学和数字媒体相结合，为妇女和少数群体提供了进入计算机科学的宝贵学习经验，否则他们可能会被传统的计算工作吓倒。该项目通过与斯托尼布鲁克大学正在进行的几项工作建立伙伴关系，包括科学与工程妇女（WISE）和路易斯·斯托克斯少数民族参与联盟（LSAMP）倡议，扩大了代表性不足的少数民族对STEM的接触。该项目研究了一种协同计算和行为方法，用于模拟人类注意力的运动。这种方法是基于这样一个假设，即对图像（或视频帧）的注意力参与取决于正在观看的像素和观看者的先前状态。基于这一假设，视觉注意力被视为一个马尔可夫决策过程，逆强化学习被用来学习奖励函数，以关联特定的时空区域在图像中，对应于像素在观众的瞬时注意力轨迹，与奖励。在这种新的方法下，注意力机制被视为一个代理，其行动是选择一个位置，在图像或图像帧，将最大限度地提高其总回报。该模型正在根据行为基础事实进行评估，该行为基础事实包括人们在自由观看和视觉搜索任务的背景下观看图像和视频时的眼球运动。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（13）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Learning Visual Emotion Representations From Web Data

DOI：
10.1109/cvpr42600.2020.01312
发表时间：
2020-06
期刊：
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
影响因子：
0
作者：
Zijun Wei;Jianming Zhang;Zhe L. Lin;Joon-Young Lee;Niranjan Balasubramanian;Minh Hoai;D. Samaras
通讯作者：
Zijun Wei;Jianming Zhang;Zhe L. Lin;Joon-Young Lee;Niranjan Balasubramanian;Minh Hoai;D. Samaras

Benchmarking Gaze Prediction for Categorical Visual Search

DOI：
10.1109/cvprw.2019.00111
发表时间：
2019-06
期刊：
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
影响因子：
0
作者：
G. Zelinsky;Zhibo Yang;Lihan Huang;Yupei Chen;Seoyoung Ahn;Zijun Wei;Hossein Adeli;D. Samaras;Minh Hoai
通讯作者：
G. Zelinsky;Zhibo Yang;Lihan Huang;Yupei Chen;Seoyoung Ahn;Zijun Wei;Hossein Adeli;D. Samaras;Minh Hoai

Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning.

使用逆增强学习来预测目标指导的人类注意力。

DOI：
10.1109/cvpr42600.2020.00027
发表时间：
2020-06
期刊：
Proceedings. IEEE Computer Society Conference on Computer Vision and Pattern Recognition
影响因子：
0
作者：
Yang Z;Huang L;Chen Y;Wei Z;Ahn S;Zelinsky G;Samaras D;Hoai M
通讯作者：
Hoai M

Patch-level Gaze Distribution Prediction for Gaze Following

DOI：
10.1109/wacv56688.2023.00094
发表时间：
2022-11
期刊：
2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
影响因子：
0
作者：
Qiaomu Miao;Minh Hoai;D. Samaras
通讯作者：
Qiaomu Miao;Minh Hoai;D. Samaras

Target-absent Human Attention

DOI：
10.48550/arxiv.2207.01166
发表时间：
2022-07
期刊：
Computer vision - ECCV ... : ... European Conference on Computer Vision : proceedings. European Conference on Computer Vision
影响因子：
0
作者：
Zhibo Yang;Sounak Mondal;Seoyoung Ahn;G. Zelinsky;Minh Hoai;D. Samaras
通讯作者：
Zhibo Yang;Sounak Mondal;Seoyoung Ahn;G. Zelinsky;Minh Hoai;D. Samaras