Character n-gram Embeddings to Improve RNN Language Models
Character n-gram Embeddings to Improve RNN Language Models
复制标题
DOI：
10.1609/aaai.v33i01.33015074
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Sho Takase;Jun Suzuki;M. Nagata
中科院分区：
文献类型：
--
作者：
Sho Takase;Jun Suzuki;M. Nagata
提出一种利用字符信息的递归神经网络(RNN)语言模型。我们在单词嵌入结构领域的研究基础上重点研究了字符n-gram(Witing等人。2016)。我们提出的方法从字符ngram嵌入构造词嵌入，并将其与普通词嵌入相结合。我们在语言建模数据集Penn Treebank、Wikitext-2和Wikitext-103上证明了所提出的方法达到了最好的困惑。此外，我们还在应用程序任务：机器翻译和标题生成上进行了实验。实验结果表明，我们提出的方法对这些任务也有积极的影响。
This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character ngram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks