A Study for Knowledge ExtractionAid System from Web Text

本研究的目标是从海量的Web页面中抽取文本或部分文本，其中包含各种用户感兴趣的知识。为此，我们开发了以下系统。(1)使用我们已经开发的术语提取系统Gensen Web来提取表征搜索引擎结果网页的术语的系统。(2)通过(1)的系统提取术语的定义以及这些术语之间的关系的系统。为了完成这一任务，我们利用了通过Web搜索引擎的使用咨询系统--猕猴桃。(3)为了使(2)的系统更高效，我们采用了后缀数组技术，并使用了预先抓取的网页。我们将该系统命名为“UT-Kiwi”，并将其从互联网上公开提供。(4)为了增强上述系统，我们开发了一个名为“Nayose”的人名搜索引擎。当我们在页面上搜索给定的人名时，我们得到的页面指示不同的人，即使他们的名字相同。针对更具创新性的知识提取，研究了基于非参数贝叶斯理论的机器学习算法。(6)进一步利用英文网页，开发了英语完形填空辅助系统Sakumon系统。

We aimed at a system that extracts texts or part of texts including knowledge which various users are interested in from huge amount of Web pages in this research. We developed the following systems for this purpose.(1) A system which extracts terms that characterize a search engine result web pages using the term extraction system "Gensen Web" which we have already developed.(2) A system which extracts definition of terms which we extract by the system of (1) and relations among these terms. To accomplish this task, we utilize the usage consultation system via Web search engine called "Kiwi."(3) In order to make more efficient system of (2), we employed a suffix array technology and use the web pages crawled in advance. We named the system as "UT-Kiwi" and made it publically available from the Internet.(4) To enhance the above described systems, we developed a people name search engine named "Nayose." When we search pages for given people name, we get pages indicating distinct person even though they have the same name. Our system clusters those web pages according to the real person.(5) Aiming at more innovative knowledge extraction, we also studied new machine learning algorithms based on non-parametric Bayes theory.(6) Utilize web page in English more, we developed the Sakumon system which is an assisting system for English cloze test using English web pages.