Stacked residual recurrent neural network with word weight for text classification
Stacked residual recurrent neural network with word weight for text classification
复制标题
DOI：
--
复制
发表时间：
2017-08
期刊：
影响因子：
--
通讯作者：
Wei Cao;Anping Song;Jinglu Hu
中科院分区：
文献类型：
--
作者：
Wei Cao;Anping Song;Jinglu Hu
神经网络，特别是循环神经网络（RNN）最近被证明可以在某些文本分类任务上提供最先进的性能。然而，大多数现有方法都假设句子中的每个单词贡献相同的重要性，这与现实世界不同。例如，如果我们进行情感分析，那么“很棒”这个词比“这部电影很棒”这句话中的任何其他单词都重要得多。鉴于这一缺陷，为了实现进一步的性能，本文提出了一种采用词权重方法的堆叠式残差 RNN，将堆叠式 RNN 扩展到具有残差网络架构的深层网络，并引入基于词权重的网络来考虑每个单词的权重。我们提出的方法能够学习句子中每个单词的高层含义，并考虑文本分类任务中每个单词的权重。实验结果表明，与最先进的方法相比，我们的方法实现了高性能。
Neural networks, and in particular recurrent neural networks (RNNs) have recently been shown to give a state-ofthe-art performance on some text classification tasks. However, most existing methods assume that each word in a sentence contributes the same importance, it is different from the real world. For example, if we do sentiment analysis, the word ”awesome” is much more important than any other words in the sentence ”This movie is awesome”. Motivated by this deficiency and in order to achieve a further performance, in this paper, a Stacked Residual RNN with Word Weight method is proposed, we extend the stacked RNN to a deep one with residual network architecture and introduce a word weight based network to consider the weight of each word. Our proposed method is able to learn high the hierarchical meaning of each word in a sentence and consider the weight of each word for text classification task. Experimental result indicates that our method achieves high performance compared with the state-of-the-art approaches.