Natural language processing and Improvement of full text retrieval

点击翻译按钮获取中文摘要

So far in the full text information retrieval, query condition has been Boolean of words or proximity of two words. The aim of our research is to make possible to use modifier-modifee relation in query to improve the precision rate. For this purpose, we develop Japanese BUNSETSU analysis system (i.e. morphological analysis system) and modifier-modifiee relation analysis system(i.e. syntactic analisys system). They are integrated into the system on PC named IBUKI. The accuracy of IBUKI is more than 99% in BUNSETSU analysis and about 90% in modifier-modifee analysis. To improve its accuracy further we are now developing compound word analysis module based on automaton formulalization and developing a new idea of function words dictionary which registers almost all function words sequence appeared in BUNSETSU. As for retrieval system, we have builet systems for patent documents, news paper articles and example sentence collection in Japanese-English dictionary. By using modifier-modifee condition, retrieval precision was shown to be improved from 84% to 96% in comparison with proximity condition.