I-Corps: Liveness detection and integrity authentication of digital audio

这个I-Corps项目更广泛的影响/商业潜力是为数字音频开发可靠的活性检测和完整性认证工具。人工智能、虚假音频生成和智能扬声器的便捷访问等领域的最新技术进步对基于语音激活设备/服务（VAD/S）的系统构成了严重的安全和隐私威胁。研究人员已经证明，现有的VAD/S容易受到攻击，并且VAD/S在电子商务、基于语音的搜索、基于扬声器识别的银行业务和远程身份验证中的日益普及凸显了这一问题。所提出的技术旨在保护VAD/S，例如，Google Home、亚马逊Alexa、苹果Siri等，抵御越来越多的攻击，包括欺骗、重放和deepfake攻击。具体地，所提出的技术可以使用针对实时系统的活性检测来可靠地检测和防止对说话者/语音识别系统的攻击，并且确定针对离线系统的完整性验证。此外，所提出的技术可能有利于其他领域，包括多媒体取证，电子发现，远程生物识别验证，执法和娱乐业。这是一个相关的问题，涉及到现代数字使用的许多关键应用程序的安全性，例如说话人验证，语音识别，金融科技，电子商务和社交媒体平台。可靠的活性检测可以用于对抗deepfakes对说话人验证的威胁以及社交媒体平台上虚假信息和虚假信息的传播，同时保护通常用于非侵入性身份验证的说话人验证系统。所提出的音频数据框架的活性检测和完整性验证依赖于声学和光声过程的物理学以及由于重放攻击而导致的失真伪影的数学建模。 该解决方案将使用这些来检测语音激活设备和服务输入的音频的活性。此外，所提出的创新利用声学环境、采集设备和后处理伪像进行取证分析和内容完整性认证。这项技术可以保护VAD/S免受越来越多的攻击，并为法医检查员和执法人员提供强大的、计算效率高的、可扩展的和可靠的工具来执行在线和离线法医分析。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

The broader impact/commercial potential of this I-Corps project is the development of a reliable liveness detection and integrity authentication tool for digital audio. Recent technological advances in the areas of artificial intelligence, fake audio generation, and easy access to smart speakers pose serious security and privacy threats to voice activated devices/services (VAD/S)-based systems. Researchers have demonstrated existing VAD/S are vulnerable to attacks and the growing adoption of VAD/S for e-commerce, voice-based search, and speaker recognition-based banking and remote identity verification highlights the problem. The proposed technology aims to secure VAD/S, e.g., Google Home, Amazon Alexa, Apple Siri, etc., against a growing number of attacks including spoofing, replay, and deepfake attacks. Specifically, the proposed technology may reliably detect and prevent attacks on speaker/speech recognition systems using liveness detection for real-time systems and determine integrity verification for off-line systems. In addition, the proposed technology may benefit other areas including multimedia forensics, e-discovery, remote biometric verification, law enforcement and the entertainment industry.This I-Corps project is based on the development of real-time liveness detection and integrity authentication of digital audio data. This is a relevant problem that concerns security of many critical applications central to modern digital usage such as speaker verification, speech recognition, fintech, e-commerce, and social medial platforms. Reliable liveness detection may be used to counter the threat of deepfakes to speaker verification and the spread of disinformation and falsehood on social media platforms while safeguarding speaker verification systems commonly used for noninvasive identity verification. The proposed liveness detection and integrity verification of audio data framework relies on the physics of acoustic and photo-acoustic processes and mathematical modeling of distortion artifacts due replay attacks. The solution will use these to detect liveness of audio with the input of voice activated devices and services. In addition, the proposed innovation leverages the acoustic environment, acquisition device, and post-processing artifacts for forensic analysis and content integrity authentication. This technology may protect VAD/S against the growing number of attacks and provide forensic examiners and law enforcement personnel with a powerful, computationally efficient, scalable, and reliable tool to perform online as well as offline forensics analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.