TEXTUAL INFORMATION RETRIEVAL TESTING

测试检索方法有效性的新方法已经出现 发展起来的。 两种新的评定统计方法有效性的方法 已经开发了不需要人工判断的检索 他们的申请。这两种方法都依赖于超几何分布 概率论。一种方法涉及两种统计方法的使用 其结果被比较的独立方法，并且由于 独立性使人能够对绝对表现得出结论。 正在研究的方法。这种统计上独立的例子 构造了成对的方法。第二种方法涉及 构建真实数据库的抽象模型，其中 对相关性关系进行建模，以便可以测试检索。 一种基于信息论的新的检索性能衡量标准 被发现了。这一措施应用起来很简单，而且可以计算出 中的排序检索方法产生的信息比特 将相关文档集中在检索的前n个等级中 手术。它有许多直观上想要的属性，并与 当后者允许进行明确的比较时，精确回想曲线 不同的检索方法。 正在进行研究，以评估提取测试对 测试集中的查询数和使用的特定度量值。 几个经典测试集(CRAN、CSI和MED等)以及 Medline在分子生物学领域的测试子集正在研究中。

New approaches to testing the effectiveness of retrieval methods have been developed. Two new methods of rating the effectiveness of statistical methods of retrieval have been developed which do not require human judgments for their application. Both methods rely on the hypergeometric distribution of probability theory. One method involves the use of two statistically independent methods whose results are compared and because of the independence allow one to draw conclusions about the absolute performance of the method under study. Examples of such statistically independent pairs of methods are constructed. The second method involves the construction of an abstract model of a real database in which the relevance relation is modelled so that retrieval may be tested. A new measure of retrieval performance based on information theory has been discovered. This measure is simple to apply and calculates the number of bits of information produced by a ranked retrieval method in concentrating relevant documents in the first n ranks in the retrieval operation. It has many intuitively desirable properties and agrees with precision-recall curves when the latter allow the unambiguous comparison of different methods of retrieval. Studies are under way to evaluate the sensitivity of retrieval testing to the number of queries in a test set and to the particular measure used. Several of the classical test sets (CRAN, CISI, and MED, etc.) as well as a test subset of Medline in the area of molecular biology are under study.