RI: Medium: Collaborative Research: Understanding and Editing Visual Sentiment

RI：媒介：协作研究：理解和编辑视觉情感

• 批准号: 1704309
• 负责人: Liqiang Wang
• 金额: $ 48.57万
• 依托单位: The University of Central Florida Board of Trustees
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-07-01 至 2023-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1704309&HistoricalAwards=false
• 关键词: RI; Medium; Collaborative; Research; Understanding

The project develops computer vision and pattern recognition technologies for visual sentiment understanding and visual sentiment editing. The interdisciplinary research team investigates the problem of understanding how images and video convey emotion. The project develops methods to infer, edit, and synthesize visual sentimental content in image/videos, in addition to their semantic contents. The project applies developed technologies to reduce violence from multimedia materials for children, and negative psychological impacts from social media for posttraumatic stress disorder (PTSD) patients. The project integrates research and education by creating new interdisciplinary courses and training graduate students. The project builds connection with the veteran academic resource center on the campus to help PTSD patients to recover from mental health problems. The research team also shares collected data with research communities.This research develops visual sentiment understanding algorithms through joint extraction of sentiments and semantics, in order to advance the understanding of how semantic entities substantiate and carry sentiments at a fine-grained object or pixel level. Computer vision algorithms and psychometric assessment techniques are combined to automatically analyze visual and recognize sentiments and emotions from multimedia materials and social media contents posted and shared by veterans. The research also explores methods of visual sentiment editing to reduce violence from multimedia materials and social media contents. The research can help (1) to protect children from accessing violent multimedia materials, and (2) to provide appropriate social media contents for applications of automatically detecting violent contents from veteran-shared multimedia.

该项目开发了计算机视觉和模式识别技术，以了解视觉情感理解和视觉情感编辑。跨学科研究团队研究了了解图像和视频如何传达情感的问题。该项目还开发了用于在图像/视频中推断，编辑和合成视觉情感内容的方法，此外它们的语义内容。该项目采用了开发的技术来减少儿童多媒体材料的暴力行为，并对创伤后应激障碍（PTSD）患者的社交媒体产生负面的心理影响。该项目通过创建新的跨学科课程和培训研究生来整合研究和教育。 该项目与校园的资深学术资源中心建立联系，以帮助PTSD患者从心理健康问题中康复。研究团队还与研究社区分享了收集的数据。这项研究通过共同提取观点和语义来开发视觉情感理解算法，以促进对语义实体如何在细粒度或像素级别上证明和携带情感的理解。将计算机视觉算法和心理测量评估技术组合在一起，以自动分析视觉效果，并认识到来自多媒体材料和社交媒体内容的情感和情感，由退伍军人发布和共享。该研究还探讨了视觉情感编辑的方法，以减少多媒体材料和社交媒体内容的暴力行为。这项研究可以帮助（1）保护儿童免受暴力多媒体材料的侵害，（2）为自动检测资深共享多媒体的暴力内容的应用提供适当的社交媒体内容。

项目成果

Learning to Adaptively Scale Recurrent Neural Networks

• DOI: 10.1609/aaai.v33i01.33013822
• 发表时间: 2019-02
• 期刊: ArXiv
• 作者: Hao Hu;Liqiang Wang;Guo-Jun Qi

WCP: Worst-Case Perturbations for Semi-Supervised Deep Learning

• DOI: 10.1109/cvpr42600.2020.00397
• 发表时间: 2020-06
• 期刊: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Liheng Zhang;Guo-Jun Qi

Generalized Loss-Sensitive Adversarial Learning with Manifold Margins

• DOI: 10.1007/978-3-030-01228-1_6
• 发表时间: 2018-09
• 期刊: Organic letters
• 影响因子: 5.2
• 作者: Marzieh Edraki;Guo-Jun Qi

AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations Rather Than Data

• DOI: 10.1109/cvpr.2019.00265
• 发表时间: 2019-01
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Liheng Zhang;Guo-Jun Qi;Liqiang Wang;Jiebo Luo

Contrastive JS: A Novel Scheme for Enhancing the Accuracy and Robustness of Deep Models

• DOI: 10.1109/tmm.2022.3232030
• 发表时间: 2023
• 期刊: IEEE Transactions on Multimedia
• 影响因子: 7.3
• 作者: Weiwei Xing;Jie Yao;Zixia Liu;Weibin Liu;Shunli Zhang;Liqiang Wang