Statistical phrase extraction techniques in natural language databases.

在自然语言文本中定位重要短语的能力对于在文本中索引或放置超链接的目的是有用的。在这两种情况下，人们都试图改善对文本材料的访问。在过去，用于定位短语的最常用的方法是词性标注器。我们开发了一种新的方法，使用一些评分算法来对短语进行排名，以确定它们的有用程度。已经开发和测试了八种不同的方法。他们已被证明是有效的，从统一的医学语言系统的国家医学图书馆开发的所有短语中获得的子集的Medline文件集合的高排名。六种方法已被结合起来，产生最佳的评分方法，并已被证明是有用的，在提取材料的质量类似于已经在UMLS。它们也似乎很有希望作为一种标记文本的超链接用于导航目的。- 统计方法短语识别超几何分布泊松分布同现

The ability to locate important phrases in natural language text is useful for the purposes of indexing or placing hyperlinks in text. In either case one seeks to improve access to the textual material. In the past the most common method used for the location of phrases has been a part of speech tagger. We have developed a new approach that uses a number of scoring algorithms to rank phrases as to how useful they may be. Eight different methods have been developed and tested. They have proved effective in ranking known phrases from the Unified Medical Language System developed by the National Library of Medicine high among all the phrases obtained from subsets of the Medline document collection. Six of the methods have been combined to produce optimal scoring methods and have proven useful in extracting material of quality similar to that already in the UMLS. They also appear promising as a way to mark text with hyperlinks for navigation purposes. - statistical methods phrase identification hyper geometric distribution Poisson distribution co-occurrence