Construction of an Intelligent System for information Retrieval in an Environment of Information Network

本课题旨在实现一个智能化的互联网信息检索系统，取得的主要成果如下：(1)基本原则采用“关键概念”的使用、检索结果相关性的估计以及作为用户界面的口语对话作为基本原则。(2)未知词的处理为了进行基于关键概念的搜索，系统必须推断出未在系统词典中注册的关键词的概念。在广泛收集和分类这些“未知词”的基础上，开发了处理因转录变化而产生的未知词以及由已知语素组成的未知复合词的方法。(3)一词多义的处理为了实现基于概念的搜索，必须同时处理关键词的一词多义和同义词。针对多义现象，提出了一种基于搭配信息的消歧方法。在同义词方面，提出了一种基于同义词概念对关键词进行扩展的方法。(4)检索结果的相关性估计开发了一种基于关键词在文档中出现的次数和位置来估计文档与查询的相关性程度的方法。对估计进行了优化，使估计的相关性得分与基于人的判断的实际得分之间的相关性最大化。(5)基于用户和系统建模的对话管理在分析模拟用户与系统对话的基础上，提出了对话管理方法。它对用户和系统分别建模，由两个有限状态自动机表示，主要通过他们的话语交换信息。结合上述结果构建了一个原型系统，并通过实验验证了其有效性。

The project aimed at realizing an intelligent system for information retrieval over the Internet, The main results obtained are as follows.(1)Basic principlesUse of 'key concepts', estimation of relevance of retrieved results, and spoken dialogue as the user interface, were adopted as basic principles.(2)Processing of unknown wordsIn order to conduct search based on key concepts, the system has to infer the concepts of key words that are not registered in the system's lexicon. On the basis of an extensive collection and classification of such 'unknown words', methods were developed for processing unknown words arising from variations of transcription as well as unknown compound words consisting of known morphemes.(3)Processing of polysemy and synonymyIn order to realize concept-based search, both polysemy and synonymy of keywords have to be coped with. As for polysemy, a method for disambiguation was developed on the basis of collocation information. As for synonymy, a method was developed to expand a keyword on the basis of its concept.(4)Relevance estimation of retrieved resultsA method was developed for estimating the degree of relevance of a document to a query on the basis of number and location of occurrence of keywords within a document. The estimation was optimized to maximize the correlation between the estimated relevance score and the actual score based on human judgment.(5)Dialogue management based on user and system modelingA method for dialogue management was developed on the basis of analysis of simulated dialogues between a user and the system. It adopts separate modeling of the user and the system, represented by two finite-state automata exchanging information mainly through their utterances.(6)Construction of a prototype systemA prototype system was constructed combining the above-mentioned results, and its validity was tested and confirmed experimentally.