SaTC: CORE: Small: Robust Speaker and Speech Recognition Under AI-Driven Physical and Digital Attacks

SaTC：核心：小型：人工智能驱动的物理和数字攻击下的鲁棒扬声器和语音识别

• 批准号: 2310207
• 负责人: Qiben Yan
• 金额: $ 60万
• 依托单位: Michigan State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2310207&HistoricalAwards=false
• 关键词: SaTC; CORE; Small; Robust; Speaker

Recent advances in artificial intelligence (AI) technologies have driven the widespread popularity and acceptance of commercial voice assistants, transforming the way in which we interact with the digital ecosystem and physical devices. These voice assistants have diverse applications across many critical sectors, ranging from smart homes and healthcare to e-commerce and self-driving vehicles. However, this rapidly evolving technology has introduced new cybersecurity threats that could significantly undermine voice system security. For example, the criminal exploitation of AI technologies has led to the advent of audio deepfakes that can be used to circumvent speaker verification protocols in banking systems. This project addresses AI-driven attacks through a comprehensive exploration of audio attack strategies and the development of a defensive framework to strengthen recognition performance and counteract these attacks. The project's broader significance and importance are underpinned by its deep understanding of AI-driven voice attacks and the methodologies to deploy defenses against them, thereby enhancing the safety of voice-controlled systems across various sectors. The project also provides research training opportunities for underrepresented students, both undergraduate and graduate, as well as K-12 students through well-crafted curriculum design and engaging outreach activities.The goal of this project is to scrutinize the robustness of automatic speaker and speech recognition (ASSR) against AI-driven digital and physical voice attacks, and further develop a defensive framework for enhancing the resilience of voice signals and voice models. The research activities include: (i) characterizing physically-realizable attacks targeting speaker recognition and measuring the impact of stealthy attacks on speech recognition; (ii) designing robust learning-based defenses by shielding the voice signals from threats posed by voice synthesis techniques, and analyzing high-frequency voice components to reveal attack signals; and (iii) developing a robustness enhancement framework that incorporates vocal styles and facial expression features in a multi-modality speaker and speech representation scheme. The insights and technologies generated through this project significantly advance our understanding of voice assistant security, foster secure next-generation voice applications, and enable voice systems to fully leverage emerging sensing and learning technologies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能（AI）技术的最新进展推动了商业语音助手的广泛普及和接受，改变了我们与数字生态系统和物理设备交互的方式。这些语音助手在许多关键领域都有不同的应用，从智能家居和医疗保健到电子商务和自动驾驶汽车。然而，这种快速发展的技术带来了新的网络安全威胁，可能会严重破坏语音系统的安全性。例如，对人工智能技术的犯罪利用导致了音频深度伪造的出现，这些音频深度伪造可用于规避银行系统中的说话人验证协议。该项目通过全面探索音频攻击策略和开发防御框架来解决AI驱动的攻击，以加强识别性能并抵御这些攻击。该项目更广泛的意义和重要性得益于其对人工智能驱动的语音攻击的深刻理解以及部署防御措施的方法，从而增强了各个部门语音控制系统的安全性。该项目还通过精心设计的课程设计和参与外展活动，为代表性不足的本科生和研究生以及K-12学生提供研究培训机会。该项目的目标是仔细检查自动说话人和语音识别（ASSR）对人工智能驱动的数字和物理语音攻击的鲁棒性，并进一步制定防御框架，以增强语音信号和语音模型的弹性。研究活动包括：（i）表征针对说话人识别的物理可实现攻击，并测量隐形攻击对语音识别的影响;（ii）通过屏蔽语音信号免受语音合成技术带来的威胁，并分析高频语音成分以揭示攻击信号，来设计基于学习的鲁棒防御;以及（iii）开发鲁棒性增强框架，其在多模态说话者和语音表示方案中结合了声音风格和面部表情特征。通过该项目产生的见解和技术大大提高了我们对语音助理安全的理解，促进了安全的下一代语音应用，并使语音系统能够充分利用新兴的传感和学习技术。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。