Neural Network Based Multi-Factor Aware Joint Training for Robust Speech Recognition
Neural Network Based Multi-Factor Aware Joint Training for Robust Speech Recognition
复制标题
基于神经网络的多因素感知联合训练，实现鲁棒语音识别
DOI：
10.1109/taslp.2016.2598308
复制
发表时间：
2016-12
影响因子：
5.4
通讯作者：
Dong Yu
中科院分区：
文献类型：
--
作者：
Yanmin Qian;Tian Tan;Dong Yu
尽管自动语音识别（ASR）已经取得了巨大进步，但在噪声环境中仍然存在显着的性能下降。本文提出了一种新颖的因素感知训练框架，称为基于神经网络的多因素感知联合训练，以提高噪声鲁棒语音识别的识别精度。这种方法是一种结构化模型，它将多个不同的功能模块集成到一个计算深度模型中。我们使用深度神经网络 (DNN) 探索和提取说话人、电话和环境因素表示，并将其集成到主 ASR DNN 中以提高分类准确性。此外，主 ASR DNN 中的隐藏激活用于改进因子提取，这反过来又有助于 ASR DNN。所有模型参数，包括 ASR DNN 和因子提取 DNN 中的参数，都是在多任务学习框架下联合优化的。与之前用于因素感知训练的传统技术不同，我们的方法不需要明确的单独阶段来进行因素提取和适应。此外，所提出的基于神经网络的多因素感知联合训练可以轻松地与使用 i 向量、噪声能量和 T60 值等显式因素的传统因素感知训练相结合，以获得额外的改进。所提出的方法在两个主要的噪声稳健任务上进行评估：AMI 单远距离麦克风任务（其中混响是主要关注点）和 Aurora4 任务（其中存在多种噪声类型）。这两个任务的实验表明，所提出的模型可以显着降低单词错误率（WER）。与这两项任务的基线相比，最佳配置的 WER 相对降低了 15% 以上。
Although great progress has been made in automatic speech recognition (ASR), significant performance degradation still exists in noisy environments. In this paper, a novel factor-aware training framework, named neural network-based multifactor aware joint training, is proposed to improve the recognition accuracy for noise robust speech recognition. This approach is a structured model which integrates several different functional modules into one computational deep model. We explore and extract speaker, phone, and environment factor representations using deep neural networks (DNNs), which are integrated into the main ASR DNN to improve classification accuracy. In addition, the hidden activations in the main ASR DNN are used to improve factor extraction, which in turn helps the ASR DNN. All the model parameters, including those in the ASR DNN and factor extraction DNNs, are jointly optimized under the multitask learning framework. Unlike prior traditional techniques for the factor-aware training, our approach requires no explicit separate stages for factor extraction and adaptation. Moreover, the proposed neural network-based multifactor aware joint training can be easily combined with the conventional factor-aware training which uses the explicit factors, such as i-vector, noise energy, and T60 value to obtain additional improvement. The proposed method is evaluated on two main noise robust tasks: the AMI single distant microphone task in which reverberation is the main concern, and the Aurora4 task in which multiple noise types exist. Experiments on both tasks show that the proposed model can significantly reduce word error rate (WER). The best configuration achieved more than 15% relative reduction in WER over the baselines on these two tasks.
登录
查看更多内容
DOI：
10.1109/icassp.2015.7178925
发表时间：
2015-04
期刊：
2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
影响因子：
--
作者：
Ritwik Giri;M. Seltzer;J. Droppo;Dong Yu
通讯作者：
Ritwik Giri;M. Seltzer;J. Droppo;Dong Yu
基于判别码的深度神经网络快速适应语音识别
DOI：
10.1109/taslp.2014.2346313
发表时间：
2014-12
期刊：
IEEE/ACM Transactions on Audio Speech and Language Processing
影响因子：
--
作者：
Abdel-Hamid, Ossama;Jiang, Hui;Dai, Lirong;Liu, Qingfeng
通讯作者：
Liu, Qingfeng
DOI：
10.1109/icassp.2014.6854661
发表时间：
2014-05
期刊：
2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
影响因子：
--
作者：
Chao Weng;Dong Yu;Shinji Watanabe;B. Juang
通讯作者：
Chao Weng;Dong Yu;Shinji Watanabe;B. Juang
DOI：
10.1109/icassp.2008.4518541
发表时间：
2008-05
期刊：
2008 IEEE International Conference on Acoustics, Speech and Signal Processing
影响因子：
--
作者：
Dong Yu;L. Deng;J. Droppo;Jian Wu;Y. Gong;A. Acero
通讯作者：
Dong Yu;L. Deng;J. Droppo;Jian Wu;Y. Gong;A. Acero
DOI：
10.1109/tasl.2012.2198059
发表时间：
2012-09
期刊：
IEEE Transactions on Audio, Speech, and Language Processing
影响因子：
--
作者：
Yongqiang Wang;M. Gales
通讯作者：
Yongqiang Wang;M. Gales