RI: Medium: Collaborative Research: Understanding and Editing Visual Sentiment

该项目开发用于视觉情感理解和视觉情感编辑的计算机视觉和模式识别技术。这个跨学科的研究小组调查了理解图像和视频如何传达情感的问题。除了图像/视频的语义内容外，该项目还开发了推断、编辑和合成视觉情感内容的方法。该项目应用先进技术，减少多媒体材料对儿童的暴力，以及社交媒体对创伤后应激障碍（PTSD）患者的负面心理影响。该项目通过创建新的跨学科课程和培养研究生，将研究和教育结合起来。该项目与校园内的老兵学术资源中心建立联系，帮助PTSD患者从心理健康问题中恢复过来。研究小组还与研究团体分享收集到的数据。本研究通过情感和语义的联合提取来开发视觉情感理解算法，以促进对语义实体如何在细粒度对象或像素级别上充实和承载情感的理解。计算机视觉算法和心理测量评估技术相结合，自动分析和识别退伍军人发布和分享的多媒体材料和社交媒体内容中的视觉和情感。研究还探讨了视觉情感编辑的方法，以减少多媒体材料和社交媒体内容中的暴力。本研究有助于(1)保护儿童不接触暴力多媒体材料；(2)为自动检测老兵共享多媒体中暴力内容的应用提供合适的社交媒体内容。

The project develops computer vision and pattern recognition technologies for visual sentiment understanding and visual sentiment editing. The interdisciplinary research team investigates the problem of understanding how images and video convey emotion. The project develops methods to infer, edit, and synthesize visual sentimental content in image/videos, in addition to their semantic contents. The project applies developed technologies to reduce violence from multimedia materials for children, and negative psychological impacts from social media for posttraumatic stress disorder (PTSD) patients. The project integrates research and education by creating new interdisciplinary courses and training graduate students. The project builds connection with the veteran academic resource center on the campus to help PTSD patients to recover from mental health problems. The research team also shares collected data with research communities.This research develops visual sentiment understanding algorithms through joint extraction of sentiments and semantics, in order to advance the understanding of how semantic entities substantiate and carry sentiments at a fine-grained object or pixel level. Computer vision algorithms and psychometric assessment techniques are combined to automatically analyze visual and recognize sentiments and emotions from multimedia materials and social media contents posted and shared by veterans. The research also explores methods of visual sentiment editing to reduce violence from multimedia materials and social media contents. The research can help (1) to protect children from accessing violent multimedia materials, and (2) to provide appropriate social media contents for applications of automatically detecting violent contents from veteran-shared multimedia.