A Study on Cluster-based Indexing of Textual Data

在这项研究中，我们提出了一个利用相似文档聚类的信息检索系统的框架和实现。该方法首先根据术语分布或术语序列匹配生成文档聚类及其代表性术语和短语。接下来，将每个文档聚类视为单个虚拟文档，创建扩展索引。在提交查询时，系统使用原始索引和扩展索引，并返回综合结果。在研究中，我们还证明了基于不同视角生成的索引可以增强检索系统的灵活性。在研究期间，我们重点研究了以下几个方面的内容：1.基于共现统计和互信息的共聚类方法。基于后缀数组的聚类方法，该方法利用由建议的重合分数3测量的文本元素的重复。基于簇的索引框架及其实现。基于快速重复聚类方法的实体识别未来的研究问题包括从目标检索的文档集中自动提取索引短语的统计和分析文本处理方法，以及用于识别引用相同真实世界实体的文本元素的方法。

In this study, we proposed a framework and implementation of an information retrieval system that utilizes clusters of similar documents. The proposed method first generates document clusters together with their representative terms and phrases based on the term distribution or term sequence match. Next, considering each document cluster as a single virtual document, an extended index is created. Upon a query submission, the system uses both the original and the extended indices and returns the integrated result. In the research, we also demonstrated that indices generated based on different viewpoints can be used to enhance the flexibility of the retrieval system.During the research period, we focused on the following research topics:1. Co-clustering method that is based on the co-occurrence statistics and mutual information2. Suffix-array based clustering method that utilizes the repetition of textual elements measured by the proposed coincidence score3. A framework of cluster-based indexing and its implementation4. Entity identification using the fast repetition-based clustering methodFuture research issues include statistical and analytical text processing methods to automatically extract index phrases from the target retrieved document set, and also methods for the identification of textual elements that refer to the same real-world entities.