SaTC: CORE: Small: Amplifying Deepfake Detection by Humans Using Cognitively-Inspired Interfaces

Deepfake(使用深度学习工具创建的人的经过处理的图像或视频)正变得越来越常见，也更有说服力。它们可能是危险的。它们可以使制造虚假新闻、欺诈、勒索和非共识的露骨形象变得更容易。打击深度假冒的一种方法可能是开发新的方法，当原始视频被篡改或完全构建时，向人类用户发出清晰的信息。这个项目探索了不同的方法来标记所谓的深度假视频，以了解哪些标签最明显，对人类用户最有说服力，并且最有可能帮助人们记住哪些视频是深度假视频。该项目团队正在评估三种不同方法发出深度假视频信号的有效性。两种使用文本或图标标记虚假视频的信号方法的灵感来自于当前事实核查界面的设计。该项目团队已经开发了第三个视觉指示器，名为深伪漫画。这些漫画放大了被操纵的视频中的人工制品，这些人工制品是由创建深伪的方式产生的。这种放大破坏了深度假视频的视觉连贯性，使其看起来更明显地被操纵了。该项目团队正在进行实验，以确认深伪漫画比未触摸的深伪漫画更容易检测到，并验证这种方法是一种有用的视觉指示器。该项目团队还在比较不同的信号发送方法，以了解哪种方法更有助于提醒人类用户视频是深度假冒的。在一项这样的研究中，项目团队通过衡量每个视觉指标是否改变了用户对视频是深度假冒的信心，来比较每个视觉指标如何改变人们对视频的主观印象。在另一项研究中，该项目团队正在比较三种信号发送方法对用户记忆的影响，方法是测量用户在收到不同视觉指示器的警报后，对哪些视频是深伪视频的记忆能力。总体而言，这些研究将帮助研究人员了解当视频被篡改时，哪些因素有助于警告人们，以及哪些因素帮助他们记住这是后来的深度假冒。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deepfakes (manipulated images or videos of people created using deep learning tools) are becoming more common and more convincing. They can be dangerous. They can make it easier to create false news, fraud, blackmail, and non-consensual explicit imagery. One way to combat deepfakes may be to develop new ways to make it clear to a human user when an original video has been manipulated or entirely constructed. This project explores different ways of labeling so-called deepfake videos to understand which kinds of labels are most visible, most convincing to a human user, and most likely to help people remember which videos are deepfakes.This project team is assessing the effectiveness of three different methods for signaling a deepfake video. Two signaling methods that use text or icons to flag a fake video are inspired by current designs in fact-checking interfaces. A third visual indicator, called Deepfake Caricatures, has been developed by the project team. These Caricatures magnify artifacts in the manipulated video, artifacts that result from the way deepfakes are created. This magnification disrupts the visual coherence of a deepfake video, making it look more obviously manipulated. The project team is performing experiments to confirm that Deepfake Caricatures are more detectable than untouched deepfakes and to validate that this method is a useful visual indicator. The project team also is comparing the different signaling methods to understand which is more helpful for alerting human users that a video is a deepfake. In one such study, the project team is comparing how each visual indicator changes people’s subjective impressions of videos by measuring whether they change the user’s confidence that a video is a deepfake. In another study, the project team is comparing the effect of the three signaling methods on users’ memories by measuring how well users remember which videos were deepfakes after being alerted with different visual indicators. Overall, these studies will help researchers understand what factors help warn people when a video has been manipulated, and what factors helps them remember that it was a deepfake later.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.