权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

RI: Medium: Collaborative Research: Understanding and Editing Visual Sentiments

RI：媒介：协作研究：理解和编辑视觉情感

基本信息

批准号：
1704337
负责人：
Jiebo Luo
金额：
$ 30.19万
依托单位：
University of Rochester
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-07-01 至 2021-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1704337&HistoricalAwards=false
关键词：
RI Medium Collaborative Research Understanding

项目摘要

The project develops computer vision and pattern recognition technologies for visual sentiment understanding and visual sentiment editing. The interdisciplinary research team investigates the problem of understanding how images and video convey emotion. The project develops methods to infer, edit, and synthesize visual sentimental content in image/videos, addition to their semantic contents. The project applies developed technologies to reduce violence from multimedia materials for children, and negative psychological impacts from social media for posttraumatic stress disorder (PTSD) patients. The project integrates research and education by creating new interdisciplinary courses and training graduate students. The project builds connection with the veteran academic resource center on the campus to help PTSD patients to recover from mental health problems. The research team also shares collected data with research communities.This research develops visual sentiment understanding algorithms through joint extraction of sentiments and semantics, in order to advance the understanding of how semantic entities substantiate and carry sentiments at a fine-grained object or pixel level. Computer vision algorithms and psychometric assessment techniques are combined to automatically analyze visual and recognize sentiments and emotions from multimedia materials and social media contents posted and shared by veterans. The research also explores methods of visual sentiment editing to reduce violence from multimedia materials and social media contents. The research can help (1) to protect children from accessing violent multimedia materials, and (2) to provide appropriate social media contents for applications of automatically detecting violent contents from veteran-shared multimedia.

该项目开发用于视觉情感理解和视觉情感编辑的计算机视觉和模式识别技术。跨学科研究团队调查了理解图像和视频如何传达情感的问题。该项目开发的方法来推断，编辑和合成图像/视频中的视觉情感内容，除了他们的语义内容。该项目应用已开发的技术，以减少儿童多媒体材料中的暴力，以及社交媒体对创伤后应激障碍患者的负面心理影响。该项目通过创建新的跨学科课程和培训研究生来整合研究和教育。该项目与校园内的资深学术资源中心建立联系，帮助创伤后应激障碍患者从心理健康问题中恢复过来。本研究通过情感和语义的联合提取，开发视觉情感理解算法，以促进对语义实体如何在细粒度对象或像素级别上充实和携带情感的理解。计算机视觉算法和心理测量评估技术相结合，自动分析视觉和识别情绪和情感的多媒体材料和社交媒体内容发布和共享的退伍军人。该研究还探讨了视觉情感编辑的方法，以减少多媒体材料和社交媒体内容中的暴力。该研究可以帮助（1）保护儿童免受暴力多媒体材料的访问，以及（2）为自动检测退伍军人共享多媒体中的暴力内容的应用程序提供适当的社交媒体内容。

项目成果

期刊论文数量（17）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Example-Guided Image Synthesis across Arbitrary Scenes using Masked Spatial-Channel Attention and Self-Supervision

DOI：
发表时间：
2020-04
期刊：
ArXiv
影响因子：
0
作者：
Haitian Zheng;Haofu Liao;Lele Chen;Wei Xiong;Tianlang Chen;Jiebo Luo
通讯作者：
Haitian Zheng;Haofu Liao;Lele Chen;Wei Xiong;Tianlang Chen;Jiebo Luo

TAP: Text-Aware Pre-training for Text-VQA and Text-Caption

DOI：
10.1109/cvpr46437.2021.00864
发表时间：
2020-12
期刊：
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
影响因子：
0
作者：
Zhengyuan Yang;Yijuan Lu;Jianfeng Wang;Xi Yin;D. Florêncio;Lijuan Wang;Cha Zhang;Lei Zhang
通讯作者：
Zhengyuan Yang;Yijuan Lu;Jianfeng Wang;Xi Yin;D. Florêncio;Lijuan Wang;Cha Zhang;Lei Zhang

Global Image Sentiment Transfer