Learning sentiment sentence representation with multiview attention model
Learning sentiment sentence representation with multiview attention model
复制标题
使用多视图注意力模型学习情感句子表示
DOI：
10.1016/j.ins.2021.05.044
复制
发表时间：
2021-06-08
影响因子：
8.1
通讯作者：
Zhang, Xuejie
中科院分区：
文献类型：
--
作者：
Zhang, You;Wang, Jin;Zhang, Xuejie
CNN、GRU 和 LSTM 等深度神经网络中的自注意力机制已被证明对于情感分析是有效的。然而，现有的注意力模型倾向于关注表达式中的单个标记或方面含义。如果文本包含不同角度的多种情感信息，现有模型将无法提取整个文本最关键、最全面的特征。在本研究中，提出了一种多视图注意力模型来学习句子表示。不是使用单一的注意力，而是使用多个视图向量来从不同的角度映射注意力。然后，采用融合门将这些多视图注意力结合起来得出结论。为了确保多视图注意力之间的差异，引入了正则化项来对损失函数添加惩罚。此外，所提出的模型可以扩展到其他文本任务，例如问题和主题，为分类提供全面的表示。在多类和多标签分类数据集上进行了比较实验。结果表明，该方法提高了先前提出的几种注意力模型的性能。 (c) 2021 Elsevier Inc. 保留所有权利。
Self-attention mechanisms in deep neural networks, such as CNN, GRU and LSTM, have been proven to be effective for sentiment analysis. However, existing attention models tend to focus on individual tokens or aspect meanings in an expression. If a text contains information on multiple sentiments from different perspectives, the existing models will fail to extract the most critical and comprehensive features of the whole text. In the present study, a multiview attention model was proposed for learning sentence representation. Instead of using a single attention, multiple view vectors were used to map the attentions from different perspectives. Then, a fusion gate was adopted to combine these multiview attentions to draw a conclusion. To ensure the differences between multiview attentions, a regularization item was introduced to add a penalty to the loss function. In addition, the proposed model can be extended to other text tasks, such as questions and topics, to provide a comprehensive representation for the classification. Comparative experiments were conducted on both multiclass and multilabel classification datasets. The results revealed that the proposed method improves the performance of several previously proposed attention models. (c) 2021 Elsevier Inc. All rights reserved.