RI: Small: Multi-View Learning of Acoustic Features for Speech Recognition Using Articulatory Measurements

本项目探索了基于声学和发音录音的多视角学习的语音识别声学特征学习技术。最近的工作表明，通过线性和非线性典型相关分析，使用这种策略可以提高识别能力，其中学习声学特征的转换，以便最大限度地提高与发音测量的相关性。以前的工作仅限于单一数据库和单一语言。这个项目的主要目标是学习任意使用者和语言的更好的通用特性，并开发改进的多视图技术。项目活动包括：学习时变预测；基于神经网络的多视图技术利用发音、视频、标签等进行“多视角”学习；有效实现;新的输入特征，如光谱时间滤波器；以及相关研究和教育的可视化工具。自动语音识别的一个关键组件是音频信号的表示，该表示封装了有用的信息，同时丢弃了噪声、说话者身份等。该项目旨在通过对语音发音器（嘴唇、舌头等）位置配对的录音进行统计分析和其他测量，自动学习改进的表示。该项目从基本的统计技术开始，并开发新的技术来应对语音和相关信号的挑战和机遇。该项目的影响超出了语音处理。多视图表示学习的应用包括神经学、气象学、化学计量学、计算机视觉和文本处理；所有这些都可以从改进的技术中受益。这项工作通过为语音技术课程和语音和其他信号的可视化工具生成材料来影响教育。

This project explores techniques for learning acoustic features for speech recognition, based on multi-view learning using acoustic and articulatory recordings. Recent work has shown recognition improvements using this strategy via linear and nonlinear canonical correlation analysis, in which transformations of acoustic features are learned so as to maximize correlation with (transformations of) articulatory measurements. Prior work has been limited to a single database and a single language.The main goals of this project are to learn better universal features for arbitrary speakers and languages and to develop improved multi-view techniques. Project activities include: learning time-varying projections; multi-view techniques based on neural networks; "many-view" learning using articulation, video, labels, etc.; efficient implementations; new input features such as spectro-temporal filters; and visualization tools for related research and education.A critical component of automatic speech recognition is a representation of the audio signal that encapsulates useful information while discarding acoustic noise, speaker identity, and so on. This project aims to automatically learn improved representations using statistical analysis of audio recordings paired with positions of the speech articulators (lips, tongue, etc.) and other measurements. The project starts with basic statistical techniques, and develops new techniques that address challenges and opportunities specific to speech and related signals.The project's impact extends beyond speech processing. Applications of multi-view representation learning include neurology, meteorology, chemometrics, computer vision, and text processing; all of these can benefit from the improved techniques. The work impacts education by generating materials for a Speech Technologies course and visualization tools for speech and other signals.