A Study on Term-Document Clustering based on a co-evolutionary framework

在这项研究中，我们提出了一个新的信息检索框架，我们称之为“基于聚类的索引”，并评估了使用实际的文档集合的有效性，所提出的计划采用同时聚类之间的文件和术语使用以前提出的“概率加权信息量”作为导航标准。其特点是，它的目的是开发和利用提取的术语和文档之间的联系，把他们作为“索引”在传统的检索系统。此外，所提出的方案可以被认为是一个适应的“共同进化框架”的遗传算法在文本检索领域，因为它首先随机启动集群的邻近条款和文件，然后，适用于局部优化所产生的集群，以处理大规模的现实世界的文档集合。在我们的研究中，我们还使用具有10，000 - 100，000个文档的测试集合来研究所提出的方法的有效性，如;从NTCIR 1中提取的学术会议论文摘要，从Mainichi和Nikkei CD-ROM数据库中提取的报纸文章，来自路透社或金融时报的英文报道。在使用文本分类任务的评估中，证实了生成的聚类的分类性能稍差，但几乎可以与支持向量机的分类性能相媲美，支持向量机被认为是文本分类的最佳分类器之一。此外，它表明该方法可以成功地提取类边界上的文档之间的关联，这是很难与传统的基于机器学习的分类方法。

In this study, we proposed a new framework of information retrieval, which we call "cluster-based indexing" , and evaluated the effectiveness using actual document collections.The proposed scheme employs simultaneous clustering between documents and terms using the previously proposed "probability weighted amount of information" as a navigation criteria. The feature is that it aims at exploiting and utilizing the extracted associations between terms and documents by treating them as 'indices' in conventional retrieval systems. Also, the proposed scheme can be considered as an adaptation of a "co-evolutionary framework" in genetic algorithms in the domain of text retrieval since it first randomly initiates clusters of neighboring terms and documents, and then, applies local optimization to the generated clusters in order to deal the large scale of real-world document collections.In our study, we also investigated the effectiveness of the proposed method using such test collections with 10,000 - 100,000 documents as ; abstracts of academic conference papers extracted from NTCIR1, newspaper articles from Mainichi and Nikkei CD-ROM databases, English stories from Reuters or Financial Times. In the evaluation using a text categorization task, it was confirmed that the categorization performance of the generated clusters was slightly worse but almost comparable to the one of Support Vector Machine, which is known to be one of the best classifier for text categorization. Furthermore, it was shown the method could successfully extract associations between documents on the class border, which is difficult with conventional machine-learning based categorization methods.