Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
复制标题
DOI：
--
复制
发表时间：
2016-02
影响因子：
6.9
通讯作者：
Rie Johnson;Tong Zhang
中科院分区：
文献类型：
--
作者：
Rie Johnson;Tong Zhang
One-hot CNN（卷积神经网络）已被证明对文本分类有效（约翰逊和张，2015）。我们将其视为一般框架的特殊情况，该框架联合训练线性模型和由“文本区域嵌入+池化”组成的非线性特征生成器。在这个框架下，我们探索了一种使用长短期记忆（LSTM）的更复杂的区域嵌入方法。LSTM可以嵌入可变（可能很大）大小的文本区域，而区域大小需要在CNN中固定。我们寻求在监督和半监督设置中有效和高效地使用LSTM。通过结合LSTM形式的区域嵌入和在未标记数据上训练的卷积层，获得了最佳结果。结果表明，在这个任务中，嵌入的文本区域，可以传达复杂的概念，是更有用的比孤立的单个单词的嵌入。我们报告的性能超过了以前的最佳结果在四个基准数据集。
One-hot CNN (convolutional neural network) has been shown to be effective for text categorization (Johnson & Zhang, 2015). We view it as a special case of a general framework which jointly trains a linear model with a non-linear feature generator consisting of `text region embedding + pooling'. Under this framework, we explore a more sophisticated region embedding method using Long Short-Term Memory (LSTM). LSTM can embed text regions of variable (and possibly large) sizes, whereas the region size needs to be fixed in a CNN. We seek effective and efficient use of LSTM for this purpose in the supervised and semi-supervised settings. The best results were obtained by combining region embeddings in the form of LSTM and convolution layers trained on unlabeled data. The results indicate that on this task, embeddings of text regions, which can convey complex concepts, are more useful than embeddings of single words in isolation. We report performances exceeding the previous best results on four benchmark datasets.