TEXTUAL INFORMATION RETRIEVAL TESTING

测试检索方法有效性的新方法已被 开发 两种新的评价统计方法有效性的方法 已经开发出不需要人类判断的检索， 它们的应用。这两种方法都依赖于 概率论一种方法涉及使用两个统计 独立的方法，其结果进行比较，因为 独立性允许人们得出关于绝对性能的结论， 研究中的方法。这种统计独立的例子 构造方法对。第二种方法涉及 构造真实的数据库的抽象模型，其中 对相关关系进行建模，以便可以测试检索。 一种基于信息论的新的检索性能度量方法， 被发现这种方法很容易应用， 由排序检索方法产生的信息位， 在检索中将相关文档集中在前n个等级中 操作它具有许多直观上令人满意的特性， 当后者允许明确的比较时，精确-召回曲线 不同的检索方法。 正在进行研究，以评估检索测试对以下方面的敏感性： 测试集中的查询数量以及所使用的特定度量。 几个经典的测试集（CRAN，CISI和MED等）以及 Medline在分子生物学领域的测试子集正在研究中。

New approaches to testing the effectiveness of retrieval methods have been developed. Two new methods of rating the effectiveness of statistical methods of retrieval have been developed which do not require human judgments for their application. Both methods rely on the hypergeometric distribution of probability theory. One method involves the use of two statistically independent methods whose results are compared and because of the independence allow one to draw conclusions about the absolute performance of the method under study. Examples of such statistically independent pairs of methods are constructed. The second method involves the construction of an abstract model of a real database in which the relevance relation is modelled so that retrieval may be tested. A new measure of retrieval performance based on information theory has been discovered. This measure is simple to apply and calculates the number of bits of information produced by a ranked retrieval method in concentrating relevant documents in the first n ranks in the retrieval operation. It has many intuitively desirable properties and agrees with precision-recall curves when the latter allow the unambiguous comparison of different methods of retrieval. Studies are under way to evaluate the sensitivity of retrieval testing to the number of queries in a test set and to the particular measure used. Several of the classical test sets (CRAN, CISI, and MED, etc.) as well as a test subset of Medline in the area of molecular biology are under study.