CAREER: Breaking the phonetic code: novel acoustic-lexical modeling techniques for robust automatic speech recognition

自发语音、重音语音和噪声语音继续给自动语音识别（ASR）技术带来重大挑战；对于这些类型的语音，ASR系统的错误率仍然高得令人无法接受。该项目建立了一个一致的框架，旨在应对所有这些条件。本文研究的语音变异性的新方法将问题视为语音信息不规范的问题之一：听者接收的某些信息子集将丢失或不确定。因此，词汇获取是一个语音密码破解问题——系统如何在每种情况下积累语音线索，从而在不完整证据的基础上识别单词？本项目采用多学科的研究方法，将语言学理论与语音识别技术相结合；语言特征的判别统计模型用于模拟语音中观察到的非线性重叠语音效应。该框架允许通过分析经过训练的系统派生出新的语言见解。教育计划促进跨学科研究（跨学科研究生研讨会），并通过在本科课程中早期引入语言技术主题和鼓励本科研究，增加未被充分代表的学生对计算机科学的参与。除了培养一种新的思考ASR发音变化的方式外，本研究更广泛的影响是为ASR和语言学社区提供协作资源，以便在教程和研讨会环境中进行讨论。在一致的框架中解决噪音、口音和说话风格问题，也将为当前系统服务不足的许多人改善ASR技术。

Spontaneous speech, accented speech, and speech in noise continue to provide automatic speech recognition (ASR) technology with significant challenges; error rates of ASR systems are still unacceptably high for these types of speech. This project establishes a consistent framework that seeks to cope with all of these conditions. The novel approach to phonetic variability investigated here views the problem as one of phonetic information underspecification: some subset of information that the listener receives will be missing or uncertain. Lexical access is thus a phonetic code-breaking problem --- how can a system accumulate phonetic cues in each of these conditions to recognize words on the basis of incomplete evidence? The research program of this project takes a multidisciplinary approach to integrating linguistic theory with speech recognition technology; discriminative statistical models of linguistic features are employed to model nonlinear, overlapping phonological effects observed in speech. The framework allows derivation of new linguistic insights through analysis of trained systems. The educational program fosters interdisciplinary research (with cross-disciplinary graduate seminars) and increases participation of underrepresented students in Computer Science by introducing language technology topics early into the undergraduate curriculum and encouraging undergraduate research. Apart from cultivating a new way of thinking about pronunciation variation for ASR, the broader impacts of this research are to provide collaborative resources for the ASR and linguistics communities to discuss in tutorial and workshop settings. Addressing noise, accent, and speaking style in a consistent framework will also improve ASR technology for many who are underserved by current systems.