Collaborative Research: SaTC: CORE: Small: Understanding how visual features of misinformation influence credibility perceptions

点击翻译按钮获取中文摘要

Today’s misinformation posts have increasingly been presented in visual formats, such as images, memes, and videos. Compared to text, visuals are processed faster, remembered better, and are more likely to be shared on social media. As technology makes image and video manipulation accessible to the masses, visual misinformation can be a significant threat to national security, social cohesion, and public health. Yet we need to know more about how specific visual features, such as color and face presence, may influence how people evaluate the credibility of such visual posts. This project offers a comprehensive understanding of how different visual elements may influence users’ perceived credibility of images and videos. The results help platforms and fact-checking agencies to detect visual misinformation, curb its diffusion, identify vulnerable user groups, and develop corrective interventions.Drawing broadly from literature in computer science, advertising, marketing, cognitive science, and communication, and using computer vision analysis, qualitative interviews, large-scale human annotation, and experiments, this research project aims to: 1) identify the specific visual features and mechanisms which may influence people’s credibility perceptions, 2) examine how these visual features interact with non-visual features (source, virality, etc) and user characteristics (partisanship, digital media literacy, etc), and 3) examine how these visual features can be effectively leveraged in misinformation correction efforts. The research team is compiling a large-scale open dataset of visual posts with human annotations. While existing misinformation datasets have largely focused on the veracity of messages, this dataset provides credibility perceptions along with other relevant outcomes such as attention, emotional reactions and aesthetic appeal. In addition, the research team is creating a website with accessible information to educate the general public about misinformation presented in images and videos, so that the public can be aware of their vulnerabilities and be more vigilant towards certain types of visual information.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.