SaTC: CORE: Small: Combating AI Synthesized Media Beyond Detection

近年来，网上虚假信息出现了令人震惊和不安的增长。一种令人不安的虚假信息形式是操纵图像/音频/视频来冒充他人。先进的人工智能技术可以以深度神经网络的形式生成逼真的操作，通常被称为深度伪造。深度造假可以被武器化，造成负面后果。尽管检测方法在基准数据集上表现出了良好的性能，但它们还不够，并且有一些局限性。该项目旨在通过积极主动的方法更有效地打击深度伪造，以根除深度伪造并保护个人免受深度伪造攻击。主动和主动方法在deepfake生成之前就生效了。主动方法不会干扰deepfake的训练或生成，而主动方法旨在破坏这些过程以防止deepfake。该项目工作及时提供了必要的技术，以减轻深度造假对网络空间和整个社会的负面影响。这个项目包括四个主要的研究活动。一是加强现有深度伪造检测方法对反取证攻击的防御。这里采用的方法是使用随机集成模型，该模型使用从无限分类模型集成中随机选择的成员处理输入。第二个活动旨在将deepfake归因于其生成模型，即恢复创建deepfake的具体方法。追踪deepfake的来源和作者这一步很重要。在第三个活动中，研究工作集中在通过污染训练数据主动向合成深度伪造添加痕迹的方法上。该项目的第四项活动是进一步研究通过使用数据中毒来破坏训练过程来阻碍深度假生成的方法。有毒的数据将导致效率降低和低质量的深度伪造。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent years have seen a startling and troubling rise of online disinformation. One disconcerting form of disinformation is the manipulation of images/audios/videos to impersonate someone else. Realistic manipulations can be generated by advanced AI technologies in the form of deep neural networks, and commonly known as deepfakes. Deepfakes can be weaponized to cause negative consequences. Although detection methods have demonstrated promising performance on benchmark datasets, they are not adequate and have several limitations. This project aims to combat deepfakes more effectively and beyond detection with active and proactive approaches to root out deepfakes and protect individuals from deepfake attacks. The active and proactive approaches take effect before the deepfake is generated. The active approach does not interfere with the training or generation of deepfake, whereas the proactive approach aims to disrupt these processes to prevent the deepfake. This project work provides timely and needed technologies to mitigate the negative impacts of deepfakes in cyberspace and society at large.This project includes four main research activities. The first is to strengthen the defense of current deepfake detection methods against anti-forensic attacks. The approach taken here is to use random ensemble models that process input with a randomly chosen member from an infinite ensemble of classification models. The second activity aims to attribute a deepfake to its generation model, i.e., recover the specific means that a deepfake is created. This step is important tracing a deepfake's origin and author. In the third activity, the research effort is focused on the methods that actively add traces to synthesized deepfakes by contaminating the training data. The fourth activity of this project further studies methods that can obstruct deepfake generation by using data poisoning to sabotage the training process. The poisoned data will lead to reduced efficiency and low-quality deepfakes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.