Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems
Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems
复制标题
利用不成对的文本数据训练端到端语音到意图系统
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
M. Picheny
中科院分区：
文献类型：
--
作者：
Yinghui Huang;H. Kuo;Samuel Thomas;Zvi Kons;Kartik Audhkhasi;Brian Kingsbury;R. Hoory;M. Picheny
训练端到端(E2E)神经网络语音到意图(S2I)系统直接从语音中提取意图需要大量标记意图的语音数据，这是耗时和昂贵的收集。用基于大量语音数据训练的ASR模型来初始化S2I模型可以缓解数据稀疏性。在本文中，我们尝试利用自然语言理解文本资源。我们实现了一个基于CTC的S2I系统，其性能与最先进的、传统的级联SLU系统相匹配。我们用不同数量的语音和文本训练数据进行了对照实验。当只有十分之一的原始数据可用时，意图分类的准确率绝对下降7.6%。假设我们有额外的文本到意图的数据(没有语音)，我们研究了两种技术来改进S2I系统：(1)转移学习，其中意图分类的声学嵌入与微调的BERT文本嵌入捆绑在一起；(2)数据增强，其中文本到意图的数据使用多说话者文本到语音系统转换为语音到意图的数据。所提出的方法恢复了由于使用有限的意图标记语音而造成的80%的性能损失。
Training an end-to-end (E2E) neural network speech-to-intent (S2I) system that directly extracts intents from speech requires large amounts of intent-labeled speech data, which is time consuming and expensive to collect. Initializing the S2I model with an ASR model trained on copious speech data can alleviate data sparsity. In this paper, we attempt to leverage NLU text resources. We implemented a CTC-based S2I system that matches the performance of a state-of-the-art, traditional cascaded SLU system. We performed controlled experiments with varying amounts of speech and text training data. When only a tenth of the original data is available, intent classification accuracy degrades by 7.6% absolute. Assuming we have additional text-to-intent data (without speech) available, we investigated two techniques to improve the S2I system: (1) transfer learning, in which acoustic embeddings for intent classification are tied to fine-tuned BERT text embeddings; and (2) data augmentation, in which the text-to-intent data is converted into speech-to-intent data using a multi-speaker text-to-speech system. The proposed approaches recover 80% of performance lost due to using limited intent-labeled speech.