A Study on Cluster-based Indexing of Textual Data

在这项研究中，我们提出了一个框架和实现的信息检索系统，利用集群的相似文件。该方法首先根据术语分布或术语序列匹配生成文档聚类及其代表性术语和短语。接下来，将每个文档聚类视为单个虚拟文档，创建扩展索引。在提交查询时，系统使用原始索引和扩展索引，并返回集成结果。在本研究中，我们还证明了基于不同视点生成的索引可以用于增强检索系统的灵活性。在研究期间，我们主要进行了以下研究：1.基于共现统计量和互信息的共聚类方法2.基于后缀数组的聚类方法，利用建议的重合score 3测量文本元素的重复。基于聚类的索引框架及其实现4.实体识别使用的快速重复为基础的聚类method-Future的研究问题包括统计和分析文本处理方法，自动提取索引短语从目标检索的文档集，也是指相同的现实世界的实体的文本元素的识别方法。

In this study, we proposed a framework and implementation of an information retrieval system that utilizes clusters of similar documents. The proposed method first generates document clusters together with their representative terms and phrases based on the term distribution or term sequence match. Next, considering each document cluster as a single virtual document, an extended index is created. Upon a query submission, the system uses both the original and the extended indices and returns the integrated result. In the research, we also demonstrated that indices generated based on different viewpoints can be used to enhance the flexibility of the retrieval system.During the research period, we focused on the following research topics:1. Co-clustering method that is based on the co-occurrence statistics and mutual information2. Suffix-array based clustering method that utilizes the repetition of textual elements measured by the proposed coincidence score3. A framework of cluster-based indexing and its implementation4. Entity identification using the fast repetition-based clustering methodFuture research issues include statistical and analytical text processing methods to automatically extract index phrases from the target retrieved document set, and also methods for the identification of textual elements that refer to the same real-world entities.