Using a stacked residual LSTM model for sentiment intensity prediction
Using a stacked residual LSTM model for sentiment intensity prediction
复制标题
使用堆叠残差 LSTM 模型进行情感强度预测
DOI：
10.1016/j.neucom.2018.09.049
复制
发表时间：
2018-12-17
期刊：
影响因子：
6
通讯作者：
Zhang, Xuejie
中科院分区：
文献类型：
--
作者：
Wang, Jin;Peng, Bo;Zhang, Xuejie
文本的情感强度表示其与积极情感的关联强度，它是0和1之间的连续实值。与极性分类相比，预测文本的情感强度可以实现更细粒度的情感分析。通过引入单词嵌入技术，最近使用深度神经模型的研究在情感强度预测方面优于现有的基于词典和回归的方法。为了获得更好的性能，神经网络的一种常见方法是添加更多的层以学习高级特征。然而，当深度增加时，网络会退化，变得更难训练。由于层间误差会累积，梯度会消失。为了解决这个问题，本文提出了一种堆叠残差LSTM模型来预测给定文本的情感强度。通过研究浅层和深层架构的性能，我们在每几层LSTM中引入一个剩余连接来构建一个8层神经网络。残余连接可以集中层梯度和传播误差。因此，它使更深的网络更容易优化。这种方法使我们能够成功地为这个任务堆叠更多层的LSTM，这可以提高现有方法的预测精度。实验结果表明，所提出的方法优于词典，回归，和传统的神经网络在以前的研究中提出的方法。(C)2018 Elsevier B.V.版权所有。
The sentiment intensity of a text indicates the strength of its association with positive sentiment, which is a continuous real-value between 0 and 1. Compared to polarity classification, predicting sentiment intensities for texts can achieve more fine-grained sentiment analyses. By introducing word embedding techniques, recent studies that use deep neural models have outperformed existing lexicon- and regression-based methods for sentiment intensity prediction. For better performance, a common way of a neural network is to add more layers in order to learn high-level features. However, when the depth increases, the network degrades and becomes more difficult to train. Since the errors between layers will be accumulated, and gradients will be vanished. To address this problem, this paper proposes a stacked residual LSTM model to predict sentiment intensity for a given text. By investigating the performances of shallow and deep architectures, we introduce a residual connection to every few LSTM layers to construct an 8-layer neural network. The residual connection can center layer gradients and propagated errors. Thus it makes the deeper network easier to optimize. This approach enables us to stack more layers of LSTM successfully for this task, which can improve the prediction accuracy of existing methods. Experimental results show that the proposed method outperforms lexicon-, regression-, and conventional NN-based methods proposed in previous studies. (C) 2018 Elsevier B.V. All rights reserved.