RI: Collaborative Research: Landmark-based Robust Speech Recognition Using Prosody-Guided Models of Speech Variability

尽管自动语音识别技术的发展取得了长足的进步，但我们还没有一个系统能够自动转录不受限制的对话语音，代表许多说话者和方言，并嵌入到不利的声学环境中。该方法应用新的高维机器学习技术，受语音产生和感知的经验和理论研究的限制，从数据中学习人类听者从语音中提取的信息结构。为此，我们将开发语音声学、发音可变性、韵律和句法的大词汇量心理现实模型，方法是导出反映人类语音产生和语音感知的知识表示，使用机器学习技术同时调整所有知识表示的参数，以便将识别器的结构风险降至最低。该团队将开发非线性声学地标检测器和模式分类器，它们集成了基于听觉的信号处理和声学语音处理，对噪声、说话人特征的变化和混响不变，并且可以从标记和未标记的数据中以半监督方式学习。此外，他们将使用可变帧速率分析，这将允许多分辨率分析，以及实施基于手势的词汇访问，使用各种训练数据。这项工作将改善人与机器之间的沟通和合作，并提高对人类如何产生和感知语音的理解。这项工作汇集了语音处理、声学语音学、韵律、手势音系学、统计模式匹配、语言建模和语音感知方面的专家团队，他们的教职员工来自工程学、计算机科学和语言学。学生和博士后研究员的支持和参与是该项目的一部分，他们从事语音建模和算法开发。最后，拟议的工作将产生一套数据库和工具，这些数据库和工具将被传播，为广大研究和教育界服务。

Despite great strides in the development of automatic speech recognition technology, we do not yet have a system with performance comparable to humans in automatically transcribing unrestricted conversational speech, representing many speakers and dialects, and embedded in adverse acoustic environments. This approach applies new high-dimensional machine learning techniques, constrained by empirical and theoretical studies of speech production and perception, to learn from data the information structures that human listeners extract from speech. To do this, we will develop large-vocabulary psychologically realistic models of speech acoustics, pronunciation variability, prosody, and syntax by deriving knowledge representations that reflect those proposed for human speech production and speech perception, using machine learning techniques to adjust the parameters of all knowledge representations simultaneously in order to minimize the structural risk of the recognizer. The team will develop nonlinear acoustic landmark detectors and pattern classifiers that integrate auditory-based signal processing and acoustic phonetic processing, are invariant to noise, change in speaker characteristics and reverberation, and can be learned in a semi-supervised fashion from labeled and unlabeled data. In addition, they will use variable frame rate analysis, which will allow for multi-resolution analysis, as well as implement lexical access based on gesture, using a variety of training data. The work will improve communication and collaboration between people and machines and also improve understanding of how human produce and perceive speech. The work brings together a team of experts in speech processing, acoustic phonetics, prosody, gestural phonology, statistical pattern matching, language modeling, and speech perception, with faculty across engineering, computer science and linguistics. Support and engagement of students and postdoctoral fellows are part of the project, engaging in speech modeling and algorithm development. Finally, the proposed work will result in a set of databases and tools that will be disseminated to serve the research and education community at large.