Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech Recognition
Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech Recognition
复制标题
基于判别码的深度神经网络快速适应语音识别
DOI：
10.1109/taslp.2014.2346313
复制
发表时间：
2014-12
期刊：
影响因子：
--
通讯作者：
Liu, Qingfeng
中科院分区：
文献类型：
--
作者：
Abdel-Hamid, Ossama;Jiang, Hui;Dai, Lirong;Liu, Qingfeng
深度神经网络（DNN）的快速自适应是深度学习中的一个重要研究课题。在本文中，我们提出了一种基于判别条件码的DNN通用自适应方案，该方案通过一组新的连接权重直接馈送到预训练DNN的各个层。此外，我们提出了几种训练方法，从训练数据中学习连接权重，以及相应的自适应方法，从每个新的测试条件的自适应数据中学习新的条件码。在这项工作中，快速自适应计划适用于监督说话人自适应语音识别的基础上，无论是帧级的交叉熵或序列级的最大互信息训练标准。我们提出了三种不同的方法来应用这种基于所谓的说话人代码的自适应方案：i）特征空间中的非线性特征归一化; ii）基于说话人代码的DNN的直接模型自适应; iii）与说话人代码的联合说话人自适应训练。我们已经评估了两个标准的语音识别任务，即TIMIT电话识别和大词汇量语音识别的总机任务中提出的自适应方法。实验结果表明，这三种方法都可以非常有效地使用少量的自适应数据来适应大型DNN模型。例如，Switchboard的结果表明，所提出的基于说话者代码的自适应方法可以实现高达8-10%的相对误差减少，每个说话者仅使用几十个自适应话语。最后，我们在使用序列训练标准进行说话人自适应之后在Switchboard中实现了非常好的性能（WER中为12.1%），这非常接近该任务中报告的最佳性能（“Deep convolutional neural networks for LVCSR，”T. N. Sainath，Proc. IEEE Acoust.，语音、信号处理、2013年）。
Fast adaptation of deep neural networks (DNN) is an important research topic in deep learning. In this paper, we have proposed a general adaptation scheme for DNN based on discriminant condition codes, which are directly fed to various layers of a pre-trained DNN through a new set of connection weights. Moreover, we present several training methods to learn connection weights from training data as well as the corresponding adaptation methods to learn new condition code from adaptation data for each new test condition. In this work, the fast adaptation scheme is applied to supervised speaker adaptation in speech recognition based on either frame-level cross-entropy or sequence-level maximum mutual information training criterion. We have proposed three different ways to apply this adaptation scheme based on the so-called speaker codes: i) Nonlinear feature normalization in feature space; ii) Direct model adaptation of DNN based on speaker codes; iii) Joint speaker adaptive training with speaker codes. We have evaluated the proposed adaptation methods in two standard speech recognition tasks, namely TIMIT phone recognition and large vocabulary speech recognition in the Switchboard task. Experimental results have shown that all three methods are quite effective to adapt large DNN models using only a small amount of adaptation data. For example, the Switchboard results have shown that the proposed speaker-code-based adaptation methods may achieve up to 8-10% relative error reduction using only a few dozens of adaptation utterances per speaker. Finally, we have achieved very good performance in Switchboard (12.1% in WER) after speaker adaptation using sequence training criterion, which is very close to the best performance reported in this task (“Deep convolutional neural networks for LVCSR,” T. N. Sainath , Proc. IEEE Acoust., Speech, Signal Process., 2013).
登录
查看更多内容
基于状态聚类的语音识别多重深度神经网络建模方法
DOI：
10.1109/taslp.2015.2392944
发表时间：
2015-04
期刊：
IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING
影响因子：
--
作者：
Hui Jiang;Li-Rong Dai;Yu Hu,;Qing-Feng Liu,
通讯作者：
Qing-Feng Liu,
DOI：
10.1109/icslp.1996.607807
发表时间：
1996-10
期刊：
Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96
影响因子：
--
作者：
T. Anastasakos;J. McDonough;R. Schwartz;J. Makhoul
通讯作者：
T. Anastasakos;J. McDonough;R. Schwartz;J. Makhoul
DOI：
10.1109/slt.2012.6424251
发表时间：
2012-12
期刊：
2012 IEEE Spoken Language Technology Workshop (SLT)
影响因子：
--
作者：
K. Yao;Dong Yu;F. Seide;Hang Su;L. Deng;Y. Gong
通讯作者：
K. Yao;Dong Yu;F. Seide;Hang Su;L. Deng;Y. Gong
DOI：
10.1109/icassp.2014.6854824
发表时间：
2014-05
期刊：
2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
影响因子：
--
作者：
Shaofei Xue;Ossama Abdel-Hamid;Hui Jiang;Lirong Dai
通讯作者：
Shaofei Xue;Ossama Abdel-Hamid;Hui Jiang;Lirong Dai
影响因子：
2.2
作者：
Jain, AK;Mao, JC;Mohiuddin, KM
通讯作者：
Mohiuddin, KM