Rank-Biased Precision for Measurement of Retrieval Effectiveness
Rank-Biased Precision for Measurement of Retrieval Effectiveness
复制标题
DOI：
10.1145/1416950.1416952
复制
发表时间：
2009-01-01
影响因子：
5.6
通讯作者：
Zobel, Justin
中科院分区：
文献类型：
--
作者：
Moffat, Alistair;Zobel, Justin
已经提出了一系列的方法来衡量信息检索系统的有效性。这些通常旨在提供相对于查询的文档排名的定量单值摘要。然而，这些措施中有许多是失败的。例如，召回率作为满意度的衡量标准并没有很好的依据，因为实际系统的用户无法判断召回率。平均精确度来自召回率，也存在同样的问题。此外，平均精度缺乏稳健实验所需的关键稳定性属性。在这篇文章中，我们引入了一个新的有效性度量，排名偏置精度，避免了这些问题。排名偏置精度来自一个简单的用户行为模型，是强大的，如果答案排名扩展到更大的深度，并允许准确量化的实验不确定性，即使只有部分相关性判断。
A range of methods for measuring the effectiveness of information retrieval systems has been proposed. These are typically intended to provide a quantitative single-value summary of a document ranking relative to a query. However, many of these measures have failings. For example, recall is not well founded as a measure of satisfaction, since the user of an actual system cannot judge recall. Average precision is derived from recall, and suffers from the same problem. In addition, average precision lacks key stability properties that are needed for robust experiments. In this article, we introduce a new effectiveness metric, rank-biased precision, that avoids these problems. Rank-biased precision is derived from a simple model of user behavior, is robust if answer rankings are extended to greater depths, and allows accurate quantification of experimental uncertainty, even when only partial relevance judgments are available.