I-Corps: Deep Forgery Detection Technology

I-Corps项目更广泛的影响/商业潜力是可能发展创新，以鉴定将在法律诉讼中用作证据的多媒体对象。被承认为法律证据的数字媒体必须符合法院对真实性、完整性和真实性的定义。此外，深度造假或多媒体内容部分或完全由计算机生成，目的是欺骗，已经发展到在某些情况下，人类无法在没有帮助的情况下进行检测。在法律公共部门，这项技术可以被法官或法院利用，也可以被警察、检察官、联邦调查局或其他政府实体用来收集证据。在私人方面，它不仅在法庭上使用，也在证词中使用，或者在电子发现过程中使用，这是一种已经用于审查电子文件（如电子邮件）的过程。这个I-Corps项目的基础是开发新技术，以可能检测视听伪造，包括用于操纵和/或伪造数字多媒体的各种深度伪造。现有的法医审查员通常不能满足刑事司法和社交媒体平台的要求；它们的范围有限，无法回答有关伪造性质的复杂问题。当使用低功率设备记录媒体和/或遭受反取证攻击时尤其如此。与依靠深度学习算法进行模式匹配来检测单个伪造（例如视频或语音深度伪造）的最先进工具不同，这一创新使用了多模态和神经符号人工智能（AI）方法，可以检测多种复杂的视听伪造，包括通过在文件和帧级别执行深度检查来检测深度伪造。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the potential development of innovations to authenticate multimedia objects to be used as evidence in legal proceedings. Digital media admitted as legal evidence must meet court-defined values for authenticity, integrity, and veracity. Moreover, deep fakes or multimedia content that has been partially or completely computer-generated with the intent to deceive has progressed to a level where detection in some cases is beyond the ability of humans to achieve unassisted. In the legal public sector, this technology may be leveraged by a judge or court or used to gather evidence by the police, prosecutor, FBI, or other government entity. On the private side, its use is not only in courtrooms but in depositions, or during eDiscovery, a process already used for vetting electronic documents like email. This I-Corps project is based on the development of new technology to possibly detect audio-visual forgeries, including various types of deep fakes used in the manipulation and/or falsification of digital multimedia. Existing Forensic Examiners typically do not satisfy the requirements of criminal justice and social media platforms; they are limited in scope and are unable to answer complex questions regarding the nature of the forgery. This is particularly true when the media is recorded using low-powered devices and/or has been subjected to an anti-forensic attack. Unlike the state-of-the-art tools which rely on deep learning algorithms for pattern matching to detect single forgeries (e.g., video or voice deep fakes), this innovation uses a multimodal and neuro-symbolic artificial intelligence (AI) approach, and could detect multiple complex audiovisual forgeries, including deep fakes by performing the deep inspection at the file- and frame- levels.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.