A Study on Cluster-based Indexing of Textual Data

点击翻译按钮获取中文摘要

In this study, we proposed a framework and implementation of an information retrieval system that utilizes clusters of similar documents. The proposed method first generates document clusters together with their representative terms and phrases based on the term distribution or term sequence match. Next, considering each document cluster as a single virtual document, an extended index is created. Upon a query submission, the system uses both the original and the extended indices and returns the integrated result. In the research, we also demonstrated that indices generated based on different viewpoints can be used to enhance the flexibility of the retrieval system.During the research period, we focused on the following research topics:1. Co-clustering method that is based on the co-occurrence statistics and mutual information2. Suffix-array based clustering method that utilizes the repetition of textual elements measured by the proposed coincidence score3. A framework of cluster-based indexing and its implementation4. Entity identification using the fast repetition-based clustering methodFuture research issues include statistical and analytical text processing methods to automatically extract index phrases from the target retrieved document set, and also methods for the identification of textual elements that refer to the same real-world entities.