Direct adaptation of hybrid DNN/HMM model for fast speaker adaptation in LVCSR based on speaker code
Direct adaptation of hybrid DNN/HMM model for fast speaker adaptation in LVCSR based on speaker code
复制标题
DOI：
10.1109/icassp.2014.6854824
复制
发表时间：
2014-05
期刊：
影响因子：
--
通讯作者：
Shaofei Xue;Ossama Abdel-Hamid;Hui Jiang;Lirong Dai
中科院分区：
文献类型：
--
作者：
Shaofei Xue;Ossama Abdel-Hamid;Hui Jiang;Lirong Dai
近年来，针对语音识别[1]中的混合DNN-HMM模型，提出了一种有效的快速说话人自适应方法。这种自适应方法依赖于对所有说话人的大型通用自适应神经网络和使用标准反向传播算法的多个小说话人代码的联合学习。在本文中，我们提出了一种在模型空间中直接自适应的替代方法，其中说话人代码通过一组新的连接权值直接连接到原始DNN模型，可以非常有效地从全部或部分训练数据中估计出来。因此，由于该方法消除了估计另一个自适应神经网络的耗时训练过程，因此更适合于大规模语音识别任务。在这项工作中，我们在大规模320小时的总机任务中评估了所提出的直接基于sc的自适应方法。实验结果表明，本文提出的基于sc的快速自适应方法不仅对小型识别任务非常有效，而且对非常大规模的任务也非常有效。例如，研究表明，所提出的方法通过每个说话者只使用非常少量的自适应话语（从10到几十），可使总机中的单词错误率相对降低8%。此外，与[1]中的方法相比，适应所需的额外训练时间也显著减少。
Recently an effective fast speaker adaptation method using discriminative speaker code (SC) has been proposed for the hybrid DNN-HMM models in speech recognition [1]. This adaptation method depends on a joint learning of a large generic adaptation neural network for all speakers as well as multiple small speaker codes using the standard back-propagation algorithm. In this paper, we propose an alternative direct adaptation in model space, where speaker codes are directly connected to the original DNN models through a set of new connection weights, which can be estimated very efficiently from all or part of training data. As a result, the proposed method is more suitable for large scale speech recognition tasks since it eliminates the time-consuming training process to estimate another adaptation neural networks. In this work, we have evaluated the proposed direct SC-based adaptation method in the large scale 320-hr Switchboard task. Experimental results have shown that the proposed SC-based rapid adaptation method is very effective not only for small recognition tasks but also for very large scale tasks. For example, it has shown that the proposed method leads to up to 8% relative reduction in word error rate in Switchboard by using only a very small number of adaptation utterances per speaker (from 10 to a few dozens). Moreover, the extra training time required for adaptation is also significantly reduced from the method in [1].