I-Corps: Liveness detection and integrity authentication of digital audio

这个i-Corps项目的更广泛的影响/商业潜力是为数字音频开发可靠的活体检测和完整性认证工具。最近在人工智能、虚假音频生成和智能扬声器轻松访问领域的技术进步对基于语音激活设备/服务(VAD/S)的系统构成了严重的安全和隐私威胁。研究人员已经证明，现有的VAD/S很容易受到攻击，并且VAD/S越来越多地被用于电子商务、基于语音的搜索、基于说话人识别的银行和远程身份验证，突显了这个问题。这项拟议的技术旨在保护VoD/S，如Google Home、Amazon Alexa、Apple Siri等，免受越来越多的攻击，包括欺骗、重播和深度假冒攻击。具体地说，所提出的技术可以使用实时系统的活跃性检测来可靠地检测和防止对说话人/语音识别系统的攻击，并确定离线系统的完整性验证。此外，拟议的技术可能会使其他领域受益，包括多媒体取证、电子发现、远程生物识别验证、执法和娱乐业。这个i-Corps项目基于数字音频数据的实时活体检测和完整性认证的开发。这是一个相关的问题，涉及到许多对现代数字使用至关重要的关键应用的安全，如说话人验证、语音识别、金融科技、电子商务和社交媒体平台。可靠的活体检测可以用来对抗深度假货对说话人验证的威胁，以及社交媒体平台上虚假信息和虚假信息的传播，同时保护通常用于非侵入性身份验证的说话人验证系统。提出的音频数据活性检测和完整性验证框架依赖于声学和光声过程的物理过程，以及重放攻击造成的失真伪影的数学建模。该解决方案将使用这些功能通过语音激活设备和服务的输入来检测音频的活跃度。此外，拟议的创新利用声学环境、采集设备和后处理人工制品进行取证分析和内容完整性验证。这项技术可以保护VAD/S免受越来越多的攻击，并为法医检查员和执法人员提供一个强大、计算高效、可扩展和可靠的工具来执行在线和离线取证分析。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of a reliable liveness detection and integrity authentication tool for digital audio. Recent technological advances in the areas of artificial intelligence, fake audio generation, and easy access to smart speakers pose serious security and privacy threats to voice activated devices/services (VAD/S)-based systems. Researchers have demonstrated existing VAD/S are vulnerable to attacks and the growing adoption of VAD/S for e-commerce, voice-based search, and speaker recognition-based banking and remote identity verification highlights the problem. The proposed technology aims to secure VAD/S, e.g., Google Home, Amazon Alexa, Apple Siri, etc., against a growing number of attacks including spoofing, replay, and deepfake attacks. Specifically, the proposed technology may reliably detect and prevent attacks on speaker/speech recognition systems using liveness detection for real-time systems and determine integrity verification for off-line systems. In addition, the proposed technology may benefit other areas including multimedia forensics, e-discovery, remote biometric verification, law enforcement and the entertainment industry.This I-Corps project is based on the development of real-time liveness detection and integrity authentication of digital audio data. This is a relevant problem that concerns security of many critical applications central to modern digital usage such as speaker verification, speech recognition, fintech, e-commerce, and social medial platforms. Reliable liveness detection may be used to counter the threat of deepfakes to speaker verification and the spread of disinformation and falsehood on social media platforms while safeguarding speaker verification systems commonly used for noninvasive identity verification. The proposed liveness detection and integrity verification of audio data framework relies on the physics of acoustic and photo-acoustic processes and mathematical modeling of distortion artifacts due replay attacks. The solution will use these to detect liveness of audio with the input of voice activated devices and services. In addition, the proposed innovation leverages the acoustic environment, acquisition device, and post-processing artifacts for forensic analysis and content integrity authentication. This technology may protect VAD/S against the growing number of attacks and provide forensic examiners and law enforcement personnel with a powerful, computationally efficient, scalable, and reliable tool to perform online as well as offline forensics analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.