Class-indexing-based term weighting for automatic text classification
Class-indexing-based term weighting for automatic text classification
复制标题
DOI：
10.1016/j.ins.2013.02.029
复制
发表时间：
2013-07
期刊：
影响因子：
--
通讯作者：
F. Ren;M. G. Sohrab
中科院分区：
文献类型：
--
作者：
F. Ren;M. G. Sohrab
以往的研究主要集中在基于文档索引和基于四个基本信息元素的自动文本分类方法上。在这项研究中，我们介绍了基于类索引的术语加权方法，并判断其效果在高维和相对低维向量空间的TF，IDF和其他五个不同的术语加权方法，被认为是基线方法。首先，我们实现了一个基于类索引的TF.IDF.ICF观测词加权方法，其中引入了逆类频率（ICF）。在实验中，我们研究了TF.IDF.ICF在路透社-21578，20个新闻组和RCV 1-v2数据集上的效果，这些数据集作为基准集合，在向量空间中对稀有术语提供了积极的区分，而在文本分类（TC）任务中对频繁术语有偏见。为此，我们对ICF函数进行了修正，实现了一种新的逆类空间密度频率（ICSδF），并生成了TF.IDF.ICSδF方法，该方法对频繁项和非频繁项具有正向区分能力。我们使用术语加权方法对三个数据集的每个类别进行了详细评估。实验结果表明，本文提出的基于类索引的TF、IDF、ICSδF词加权方法优于现有的基线词加权方法。
Most of the previous studies related on different term weighting emphasize on the document-indexing-based and four fundamental information elements-based approaches to address automatic text classification (ATC). In this study, we introduce class-indexing-based term-weighting approaches and judge their effects in high-dimensional and comparatively low-dimensional vector space over the TF.IDF and five other different term weighting approaches that are considered as the baseline approaches. First, we implement a class-indexing-based TF.IDF.ICF observational term weighting approach in which the inverse class frequency (ICF) is incorporated. In the experiment, we investigate the effects of TF.IDF.ICF over the Reuters-21578, 20 Newsgroups, and RCV1-v2 datasets as benchmark collections, which provide positive discrimination on rare terms in the vector space and biased against frequent terms in the text classification (TC) task. Therefore, we revised the ICF function and implemented a new inverse class space density frequency (ICSδF), and generated the TF.IDF.ICSδF method that provides a positive discrimination on infrequent and frequent terms. We present detailed evaluation of each category for the three datasets with term weighting approaches. The experimental results show that the proposed class-indexing-based TF.IDF.ICSδF term weighting approach is promising over the compared well-known baseline term weighting approaches.