SaTC: CORE: Small: Amplifying Deepfake Detection by Humans Using Cognitively-Inspired Interfaces

点击翻译按钮获取中文摘要

Deepfakes (manipulated images or videos of people created using deep learning tools) are becoming more common and more convincing. They can be dangerous. They can make it easier to create false news, fraud, blackmail, and non-consensual explicit imagery. One way to combat deepfakes may be to develop new ways to make it clear to a human user when an original video has been manipulated or entirely constructed. This project explores different ways of labeling so-called deepfake videos to understand which kinds of labels are most visible, most convincing to a human user, and most likely to help people remember which videos are deepfakes.This project team is assessing the effectiveness of three different methods for signaling a deepfake video. Two signaling methods that use text or icons to flag a fake video are inspired by current designs in fact-checking interfaces. A third visual indicator, called Deepfake Caricatures, has been developed by the project team. These Caricatures magnify artifacts in the manipulated video, artifacts that result from the way deepfakes are created. This magnification disrupts the visual coherence of a deepfake video, making it look more obviously manipulated. The project team is performing experiments to confirm that Deepfake Caricatures are more detectable than untouched deepfakes and to validate that this method is a useful visual indicator. The project team also is comparing the different signaling methods to understand which is more helpful for alerting human users that a video is a deepfake. In one such study, the project team is comparing how each visual indicator changes people’s subjective impressions of videos by measuring whether they change the user’s confidence that a video is a deepfake. In another study, the project team is comparing the effect of the three signaling methods on users’ memories by measuring how well users remember which videos were deepfakes after being alerted with different visual indicators. Overall, these studies will help researchers understand what factors help warn people when a video has been manipulated, and what factors helps them remember that it was a deepfake later.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.