Modified frequency-based term weighting schemes for text classification
Modified frequency-based term weighting schemes for text classification
复制标题
DOI：
10.1016/j.asoc.2017.04.069
复制
发表时间：
2017-09
期刊：
影响因子：
--
通讯作者：
Thabit Sabbah;A. Selamat;Md. Hafiz Selamat;Fawaz S. Al-Anzi;E. Herrera-Viedma;O. Krejcar;H. Fujita
中科院分区：
文献类型：
--
作者：
Thabit Sabbah;A. Selamat;Md. Hafiz Selamat;Fawaz S. Al-Anzi;E. Herrera-Viedma;O. Krejcar;H. Fujita
随着互联网上文本内容的快速增长，自动文本分类是信息组织和知识管理中相对更有效的解决方案。特征选择是基于统计的文本分类的基本阶段之一，关键取决于术语加权方法。为了提高文本分类的性能，本文提出了四种改进的基于频率的术语加权方案： mTF、mTFIDF、TFmIDF 和 mTFmIDF。所提出的术语加权方案在计算现有术语的权重时考虑了缺失术语的数量。所提出的方案显示了 SVM 分类器的最高性能，微平均 F1 分类性能值为 97%。此外，在Reuters-21578、20Newsgroups和WebKB文本分类数据集上使用不同的分类算法（例如SVM和KNN）进行的基准测试结果表明，所提出的方案mTF、mTFIDF和mTFmIDF优于其他加权方案（例如TF、TFIDF和Entropy）。此外，统计显着性测试显示基于修改方案的分类性能显着增强。
With the rapid growth of textual content on the Internet, automatic text categorization is a comparatively more effective solution in information organization and knowledge management. Feature selection, one of the basic phases in statistical-based text categorization, crucially depends on the term weighting methods In order to improve the performance of text categorization, this paper proposes four modified frequency-based term weighting schemes namely; mTF, mTFIDF, TFmIDF, and mTFmIDF. The proposed term weighting schemes take the amount of missing terms into account calculating the weight of existing terms. The proposed schemes show the highest performance for a SVM classifier with a micro-average F1 classification performance value of 97%. Moreover, benchmarking results on Reuters-21578, 20Newsgroups, and WebKB text-classification datasets, using different classifying algorithms such as SVM and KNN show that the proposed schemes mTF, mTFIDF, and mTFmIDF outperform other weighting schemes such as TF, TFIDF, and Entropy. Additionally, the statistical significance tests show a significant enhancement of the classification performance based on the modified schemes.