Short text classification based on strong feature thesaurus
Short text classification based on strong feature thesaurus
复制标题
DOI：
10.1631/jzus.c1100373
复制
发表时间：
2012-09
期刊：
影响因子：
--
通讯作者：
Bing-kun Wang;Yongfeng Huang;Wanxia Yang;Xing Li
中科院分区：
文献类型：
--
作者：
Bing-kun Wang;Yongfeng Huang;Wanxia Yang;Xing Li
短文本的显着特征——数据稀疏，一直被认为是统计方法对短文本分类准确率低的主要原因。过去十年来，人们在这一领域进行了深入的研究。然而，大多数研究人员没有注意到，忽略某些特征术语的语义重要性也可能导致分类准确性较低。在本文中，我们提出了一种解决该问题的新方法，即基于潜在狄利克雷分配（LDA）和信息增益（IG）模型构建强大的特征词库（SFT）。通过在SFT中赋予特征项较大的权重，可以提高分类精度。具体来说，我们的方法通过更详细的分类似乎更有效。在两个短文本数据集中的实验表明，与包括支持向量机（SVM）和朴素贝叶斯多项式在内的最先进方法相比，我们的方法取得了改进。
Data sparseness, the evident characteristic of short text, has always been regarded as the main cause of the low accuracy in the classification of short texts using statistical methods. Intensive research has been conducted in this area during the past decade. However, most researchers failed to notice that ignoring the semantic importance of certain feature terms might also contribute to low classification accuracy. In this paper we present a new method to tackle the problem by building a strong feature thesaurus (SFT) based on latent Dirichlet allocation (LDA) and information gain (IG) models. By giving larger weights to feature terms in SFT, the classification accuracy can be improved. Specifically, our method appeared to be more effective with more detailed classification. Experiments in two short text datasets demonstrate that our approach achieved improvement compared with the state-of-the-art methods including support vector machine (SVM) and Naïve Bayes Multinomial.