Statistical phrase extraction techniques in natural language databases.

在自然语言文本中定位重要短语的能力对于在文本中建立索引或放置超链接非常有用。在任何一种情况下，人们都力求改善对文本材料的获取。在过去，最常用的短语定位方法是词性标注。我们已经开发了一种新的方法，它使用了许多评分算法来对短语的有用程度进行排名。已经开发和测试了八种不同的方法。事实证明，它们有效地将国家医学图书馆开发的统一医学语言系统中的已知短语排在Medline文档集合子集中获得的所有短语的前列。其中六种方法已被结合以产生最佳评分方法，并已被证明在提取质量类似于已经在UMLS中的材料方面是有用的。它们也很有希望作为一种用超链接标记文本的导航方式。-统计学方法短语识别超几何分布泊松分布共现

The ability to locate important phrases in natural language text is useful for the purposes of indexing or placing hyperlinks in text. In either case one seeks to improve access to the textual material. In the past the most common method used for the location of phrases has been a part of speech tagger. We have developed a new approach that uses a number of scoring algorithms to rank phrases as to how useful they may be. Eight different methods have been developed and tested. They have proved effective in ranking known phrases from the Unified Medical Language System developed by the National Library of Medicine high among all the phrases obtained from subsets of the Medline document collection. Six of the methods have been combined to produce optimal scoring methods and have proven useful in extracting material of quality similar to that already in the UMLS. They also appear promising as a way to mark text with hyperlinks for navigation purposes. - statistical methods phrase identification hyper geometric distribution Poisson distribution co-occurrence