A Study for Knowledge ExtractionAid System from Web Text

本研究的目标是从大量的网页中抽取出包含各种用户感兴趣的知识的文本或部分文本。我们为此开发了以下系统。(1)一种系统，其使用我们已经开发的术语提取系统“Gensen Web”来提取表征搜索引擎结果网页的术语。(2)一种系统，其提取我们通过（1）的系统提取的术语的定义和这些术语之间的关系。为了完成这项任务，我们利用使用咨询系统，通过Web搜索引擎称为“猕猴桃。（3）为了使（2）的系统更加高效，我们采用了后缀数组技术，并使用了预先抓取的网页。我们将该系统命名为“UT-Kiwi”，并将其从互联网上免费提供。(4)为了增强上述系统，我们开发了一个名为“Nayose”的人名搜索引擎。“当我们搜索给定人名的页面时，我们会得到指示不同人的页面，即使他们有相同的名字。我们的系统根据真实的人对这些网页进行聚类。(5)为了更好地进行知识抽取，我们还研究了基于非参数贝叶斯理论的机器学习算法。(6)为了更好地利用英文网页，我们开发了一个基于英文网页的英语完形填空辅助系统Sakumon。

We aimed at a system that extracts texts or part of texts including knowledge which various users are interested in from huge amount of Web pages in this research. We developed the following systems for this purpose.(1) A system which extracts terms that characterize a search engine result web pages using the term extraction system "Gensen Web" which we have already developed.(2) A system which extracts definition of terms which we extract by the system of (1) and relations among these terms. To accomplish this task, we utilize the usage consultation system via Web search engine called "Kiwi."(3) In order to make more efficient system of (2), we employed a suffix array technology and use the web pages crawled in advance. We named the system as "UT-Kiwi" and made it publically available from the Internet.(4) To enhance the above described systems, we developed a people name search engine named "Nayose." When we search pages for given people name, we get pages indicating distinct person even though they have the same name. Our system clusters those web pages according to the real person.(5) Aiming at more innovative knowledge extraction, we also studied new machine learning algorithms based on non-parametric Bayes theory.(6) Utilize web page in English more, we developed the Sakumon system which is an assisting system for English cloze test using English web pages.