EAGER: SaTC: SAVED: Secure Audio and Video Data from Deepfake Attacks Leveraging Environmental Fingerprints

EAGER：SaTC：SAVED：利用环境指纹保护音频和视频数据免遭 Deepfake 攻击

• 批准号: 2039342
• 负责人: Yu Chen
• 金额: $ 25.7万
• 依托单位: SUNY at Binghamton
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2039342&HistoricalAwards=false
• 关键词: EAGER; SaTC; SAVED; Secure; Audio

The fast development of artificial intelligence (AI) and machine learning algorithms is escalating the technology that empowers the ability to distort reality. It has taken an exponential leap forward to deepfake attacks, which create audio and video of real people saying and doing things they never said or did. It is ever more realistic and increasingly resistant to detection. Deepfaked video, audio, or photos published on social media platforms are highly disturbing and able to mislead the public, raising further challenges in policy, technology, social, and legal aspects. Today's deepfake tools allow people to become anyone, from Elon Musk to Eminem, during a video chat. Recent deepfake video attacks on some public scenarios have raised more concerns. Disinformation may actually cause a disturbance in our society and ruin the foundation of trust. Government agencies like the U.S. Defense Advanced Research Projects Agency (DARPA) are concerned about losing the war against deepfake attacks that use the popular machine learning technique to automatically incorporate artificial components into existing video streams. The detailed technical routines and countermeasures against deepfake attacks have not been well investigated, leaving alone a potentially effective approach to tackle the emerging threats online in real-time.This project introduces a novel solution to secure audio and video data streams against deepfake attacks. Instead of engaging in the endless AI arm races that fight fire with fire, where new machine learning algorithms keep making fake audio and video more real, this project tackles the challenging problem out of the box based on a key observation. Every audio or video stream has unique environmental fingerprints, e.g. the Electrical Network Frequency (ENF) signals, embedded when it was generated. The environmental fingerprints are random signals, which are unique, unpredictable, and unrepeatable. This project will investigate three typical application scenarios: (1) an accurate detection of deepfaked AVS data uploaded on the Internet, like social media posts; (2) an instant and accurate detection of false AVS injection attacks against online, real-time applications, like teleconferencing; and (3) a lightweight but robust version that fits on the Internet of Video Things applications, like smart public safety surveillance, which requires instant decision-making at the network edge. In addition, this project will gain deeper insights into the characteristics of the environmental fingerprints taking an information theory approach. The success of this research will deliver a disruptive technology that enables the ultimate win of the battle against the deepfake attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能（AI）和机器学习算法的快速发展正在升级技术，使其能够扭曲现实。它已经采取了指数级的飞跃，以deepfake攻击，它创建了真实的人说和做他们从未说过或做过的事情的音频和视频。它比以往任何时候都更真实，越来越难以被发现。在社交媒体平台上发布的深度伪造的视频、音频或照片非常令人不安，并可能误导公众，从而在政策、技术、社会和法律的方面带来进一步的挑战。今天的deepfake工具允许人们在视频聊天中成为任何人，从埃隆马斯克到阿姆。最近对一些公共场景的Deepfake视频攻击引发了更多担忧。虚假信息实际上可能会在我们的社会中造成混乱，破坏信任的基础。像美国国防高级研究计划局（DARPA）这样的政府机构担心会输掉针对Deepfake攻击的战争，Deepfake攻击使用流行的机器学习技术自动将人工组件纳入现有的视频流。针对deepfake攻击的详细技术例程和对策尚未得到很好的研究，更不用说实时解决在线新兴威胁的潜在有效方法了。该项目介绍了一种新颖的解决方案，以保护音频和视频数据流免受deepfake攻击。这个项目不是参与无休止的人工智能军备竞赛，以火灭火，新的机器学习算法不断使虚假的音频和视频变得更加真实的，而是根据一个关键的观察来解决这个具有挑战性的问题。每个音频或视频流都具有独特的环境指纹，例如在生成时嵌入的电网频率（ENF）信号。环境指纹是一种随机信号，具有唯一性、不可预测性和不可重复性。该项目将研究三种典型的应用场景：（1）准确检测互联网上上传的deepfaked AVS数据，如社交媒体帖子;（2）即时准确检测针对在线实时应用的虚假AVS注入攻击，如电话会议;以及（3）一个轻量级但强大的版本，适合视频物联网应用，如智能公共安全监控，这需要在网络边缘进行即时决策。此外，该项目将采用信息论方法更深入地了解环境指纹的特征。这项研究的成功将提供一种颠覆性的技术，使我们能够最终赢得与deepfake攻击的斗争。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Fairledger: a Fair Proof-of-Sequential-Work based Lightweight Distributed Ledger for IoT Networks

Fairledger：基于公平顺序工作证明的物联网网络轻量级分布式账本

• DOI: 10.1109/blockchain55522.2022.00055
• 发表时间: 2022
• 期刊: the 5th IEEE International Conference on Blockchain (Blockchain 2022
• 作者: Xu, Ronghua;Chen, Yu
• 通讯作者: Chen, Yu

DeFakePro: Decentralized Deepfake Attacks Detection Using ENF Authentication

• DOI: 10.1109/mitp.2022.3172653
• 发表时间: 2022-07
• 期刊: IT Professional
• 影响因子: 2.6
• 作者: Deeraj Nagothu;Ronghua Xu;Yu Chen;E. Blasch;Alexander J. Aved

Detecting Compromised Edge Smart Cameras using Lightweight Environmental Fingerprint Consensus

使用轻量级环境指纹共识检测受损的边缘智能相机

• DOI: 10.1145/3485730.3493684
• 发表时间: 2021
• 期刊: The 19th ACM Conference on Embedded Networked Sensor Systems
• 作者: Nagothu, Deeraj;Xu, Ronghua;Chen, Yu;Blasch, Erik;Aved, Alexander
• 通讯作者: Aved, Alexander

Deterring Deepfake Attacks with an Electrical Network Frequency Fingerprints Approach

• DOI: 10.3390/fi14050125
• 发表时间: 2022-04
• 期刊: Future Internet
• 影响因子: 3.4
• 作者: Deeraj Nagothu;Ronghua Xu;Yu Chen;E. Blasch;Alexander J. Aved

DEMA: decentralized electrical network frequency map for social media authentication

DEMA：用于社交媒体认证的去中心化电网频率图

• DOI: 10.1117/12.2663303
• 发表时间: 2023
• 期刊: Disruptive Technologies in Information Sciences VII
• 作者: Nagothu, Deeraj;Xu, Ronghua;Chen, Yu
• 通讯作者: Chen, Yu