Recurrent neural networks for language understanding
Recurrent neural networks for language understanding
复制标题
DOI：
10.21437/interspeech.2013-569
复制
发表时间：
2013-08
期刊：
影响因子：
--
通讯作者：
K. Yao;G. Zweig;M. Hwang;Yangyang Shi;Dong Yu
中科院分区：
文献类型：
--
作者：
K. Yao;G. Zweig;M. Hwang;Yangyang Shi;Dong Yu
递归神经网络语言模型（RNN-LM）最近在各种应用中表现出卓越的性能。在本文中，我们修改的架构进行语言理解，并提出了国家的最先进的广泛使用的ATIS数据集。我们方法的核心是像标准RNN-LM一样将单词作为输入，然后在输出端预测槽标签而不是单词。我们提出了几种不同的变化，在输入端使用的单词上下文的量，并在使用非词汇功能。值得注意的是，我们最简单的模型产生了最先进的结果，我们通过使用袋词，词嵌入，命名实体，句法和词类特征来推进最先进的结果。分析表明，上级性能归因于特定任务的词汇表征学习的RNN。
Recurrent Neural Network Language Models (RNN-LMs) have recently shown exceptional performance across a variety of applications. In this paper, we modify the architecture to perform Language Understanding, and advance the state-of-the-art for the widely used ATIS dataset. The core of our approach is to take words as input as in a standard RNN-LM, and then to predict slot labels rather than words on the output side. We present several variations that differ in the amount of word context that is used on the input side, and in the use of non-lexical features. Remarkably, our simplest model produces state-of-the-art results, and we advance state-of-the-art through the use of bagof-words, word embedding, named-entity, syntactic, and wordclass features. Analysis indicates that the superior performance is attributable to the task-specific word representations learned by the RNN.