SaTC: CORE: Small: Robust Speaker and Speech Recognition Under AI-Driven Physical and Digital Attacks

人工智能（AI）技术的最新进展推动了商业语音助手的广泛普及和接受，改变了我们与数字生态系统和物理设备的互动方式。这些语音助手在许多关键领域都有不同的应用，从智能家居和医疗保健到电子商务和自动驾驶汽车。然而，这种快速发展的技术带来了新的网络安全威胁，可能会严重破坏语音系统的安全性。例如，对人工智能技术的犯罪利用导致了音频深度伪造的出现，这些音频深度伪造可以用来绕过银行系统中的说话人验证协议。该项目通过全面探索音频攻击策略和开发防御框架来解决人工智能驱动的攻击，以加强识别性能并抵消这些攻击。该项目更广泛的意义和重要性得益于其对人工智能驱动的语音攻击和部署防御方法的深刻理解，从而增强了各个部门语音控制系统的安全性。该项目还通过精心设计的课程和引人入胜的外展活动，为代表性不足的本科生和研究生以及K-12学生提供研究培训机会。该项目的目标是仔细检查自动说话者和语音识别（ASSR）对人工智能驱动的数字和物理语音攻击的鲁棒性，并进一步开发一个防御框架，以增强语音信号和语音模型的弹性。研究工作包括：(1)描述针对说话人识别的物理可实现攻击的特征，并测量隐形攻击对语音识别的影响；（ii）通过屏蔽语音信号免受语音合成技术的威胁，并分析高频语音成分以揭示攻击信号，设计健壮的基于学习的防御；（iii）开发一个鲁棒性增强框架，该框架在多模态说话者和语音表示方案中结合了声音风格和面部表情特征。通过该项目产生的见解和技术显著提高了我们对语音助理安全性的理解，促进了安全的下一代语音应用，并使语音系统能够充分利用新兴的传感和学习技术。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in artificial intelligence (AI) technologies have driven the widespread popularity and acceptance of commercial voice assistants, transforming the way in which we interact with the digital ecosystem and physical devices. These voice assistants have diverse applications across many critical sectors, ranging from smart homes and healthcare to e-commerce and self-driving vehicles. However, this rapidly evolving technology has introduced new cybersecurity threats that could significantly undermine voice system security. For example, the criminal exploitation of AI technologies has led to the advent of audio deepfakes that can be used to circumvent speaker verification protocols in banking systems. This project addresses AI-driven attacks through a comprehensive exploration of audio attack strategies and the development of a defensive framework to strengthen recognition performance and counteract these attacks. The project's broader significance and importance are underpinned by its deep understanding of AI-driven voice attacks and the methodologies to deploy defenses against them, thereby enhancing the safety of voice-controlled systems across various sectors. The project also provides research training opportunities for underrepresented students, both undergraduate and graduate, as well as K-12 students through well-crafted curriculum design and engaging outreach activities.The goal of this project is to scrutinize the robustness of automatic speaker and speech recognition (ASSR) against AI-driven digital and physical voice attacks, and further develop a defensive framework for enhancing the resilience of voice signals and voice models. The research activities include: (i) characterizing physically-realizable attacks targeting speaker recognition and measuring the impact of stealthy attacks on speech recognition; (ii) designing robust learning-based defenses by shielding the voice signals from threats posed by voice synthesis techniques, and analyzing high-frequency voice components to reveal attack signals; and (iii) developing a robustness enhancement framework that incorporates vocal styles and facial expression features in a multi-modality speaker and speech representation scheme. The insights and technologies generated through this project significantly advance our understanding of voice assistant security, foster secure next-generation voice applications, and enable voice systems to fully leverage emerging sensing and learning technologies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.