EAGER: How does deep learning improve speech recognition accuracy?

普遍而准确的自动语音识别有可能以许多积极的方式改变社会，其中最重要的是为那些难以甚至不可能使用键盘与计算机交互的人提供更好的信息访问：例如老年人，身体残疾者或视力受损者。 每天都有数百万人使用基于这项技术的应用程序来解决问题，这些问题最自然地通过语音与机器交互来完成。 然而，这些应用程序中最成功的一直是相当有限的范围，因为，虽然有用，语音识别可能是令人沮丧的不可靠。例如，尽管在拥挤的房间里有很大的背景噪音，电话信道上有严重的失真，或者在他们共同的语言中口音有很大的变化，但人类很容易能够相互理解，但是即使这些问题的温和得多的例子也会完全破坏语音识别系统。 EARLY探索性研究资助（EAGER）支持一个项目，其短期目标是以深入，定量的方式了解为什么几乎所有语音识别器中使用的方法如此脆弱。 长期目标是通过开发更不脆弱的方法来利用这种理解，从而实现更准确的语音识别，并具有更广泛的适用性。这项探索性研究将首先发现为什么多层感知器（MLP）有时可以提高语音识别的准确性，其次，使用这些诊断见解来选择更好的MLP架构，第三，发布软件，以便其他人可以利用开发的方法。在过去的十年中，MLP的使用已经出现了显着的复苏，特别是最近开发的“深度”架构。 在语音识别领域，MLP有两个应用显著提高了大词汇量语音识别的准确率。 这些应用程序中的每一个都在标准语音识别机器内工作，该机器使用隐马尔可夫模型（Hidden Markov Models，HMF）来对声学、用于模型的输入（特征）的梅尔频率倒谱系数（MFCC）以及用于隐藏状态的边缘分布的多变量正态进行建模。 第一个应用程序通过使用MLP从数据中学习的新特征来增强标准模型输入，从而对标准机器进行相对较小的调整。 第二个应用程序对标准机制进行了更实质性的改变，用一个模拟边缘状态后验的MLP来代替隐藏状态边缘分布的集合。 探索性研究的目的是揭示基于MLP的特征对HMM语音识别准确率的影响机制。 这项研究建立在ICSI实验室以前的工作基础上，ICSI实验室使用模拟和一种新的采样过程来量化主要HMM假设对语音识别准确性的影响。如何实际提高语音识别的准确性，或者理论上的渐近结果。 虽然这项研究显然很重要，但它在很大程度上是通过试验和错误来进行的，特别是，它没有解决围绕MLP的这些应用如何实际提高语音识别准确性的有趣的科学问题。 对后一个问题的深入理解，在短期内，将导致语音识别准确性的进一步提高，从长远来看，将能够开发出比HMM更合适和更成功的语音识别模型，这将是该领域的一个变革性进步。

Pervasive and accurate automatic speech recognition has the potential to transform society in many positive ways, not the least of which is providing better access to information for those who find it difficult or even impossible to interact with computers using a keyboard: e.g. the elderly, the physically disabled, or the vision impaired. Every day millions of people use applications based on this technology to solve problems that are most naturally accomplished by interacting with machines via voice. However, the most successful of these applications have always been rather limited in scope, because, although useful, speech recognition can be frustratingly unreliable. For example, human beings are easily able to understand one another despite loud background noise in a crowded room, severe distortion over a telephone channel, or wide variation in accents within their common language, but even much milder examples of these problems will completely derail a speech recognition system. This EArly Grant for Exploratory Research (EAGER) supports a project whose short term goal is to understand in a deep, quantitative way why methodology used in nearly all speech recognizers is so brittle. The long term goal is to leverage this understanding by developing less brittle methodology that will enable more accurate speech recognition with a wider scope of applicability. This exploratory study will, first, discover why multilayer perceptrons (MLPs) can sometimes improve speech recognition accuracy, second, use these diagnostic insights to select better MLP architectures, and, third, release software so that others can leverage the developed methods. The use of MLPs has staged a remarkable resurgence in the last decade, in particular the "deep" architectures developed recently. In the field of speech recognition, there are two applications of MLPs that have significantly improved large vocabulary speech recognition accuracy. Each of these applications work within the standard speech recognition machinery, which uses hidden Markov models (HMMs) to model the acoustics, mel-frequency cepstral coefficients (MFCCs) for the models' inputs (features), and multivariate normals for the hidden states' marginal distributions. The first application makes a relatively minor adjustment to the standard machinery by augmenting the standard model inputs with new features learned from data using a MLP. The second application makes a more substantial change to the standard machinery by replacing the collection of hidden states' marginal distributions with a single MLP that models the marginal state posteriors. The research in the exploratory study will discover the basic mechanisms that the MLP-based features use to substantially improve HMM-based speech recognition accuracy. This research builds upon the previous work in the ICSI lab that used simulation and a novel sampling process to quantify the impacts that the major HMM assumptions have on speech recognition accuracy.Other recent research on MLPs for speech recognition has either concentrated on implementation, i.e., how to actually improve speech recognition accuracy, or on theoretical asymptotic results. While this research is obviously important, it has proceeded largely by trial and error and, in particular, it has not addressed the interesting scientific questions surrounding how these applications of MLPs actually improve speech recognition accuracy. A deeper understanding of this latter question should, in the short term, lead to further improvements in speech recognition accuracy and, in the long term, enable the development of more suitable and successful models for speech recognition than the HMM, which would be a transformative advance in the field.