Analysis and Evaluation of Measures of Information Retrieval Performance

搜索引擎和其他信息检索技术在数字时代至关重要。本研究的目标是研究分析和有效评估检索性能的新框架，着眼于促进和使研究能够导致更好的搜索引擎和其他检索技术。本文提出了两个新的框架：(1)一个信息论框架，在这个框架中，人们可以量化地评估检索性能的各种度量；(2)一个统计框架，在这个框架中，人们可以有效地估计这些检索性能的度量。前者为检索评价和分析提供了理论基础；后者为大规模有效地评估搜索引擎提供了一种实用的方法。每一个都将促进和促进研究，从而带来更好的搜索引擎和搜索技术。这个项目的潜在影响是多方面的。从研究和基础设施的角度来看，该项目将产生已发表的研究结果和免费可用的软件工件（通过http://www.ccs.neu.edu/home/jaa/IIS-0534482/），这将允许以最小的努力进行大规模检索评估。学者和其他技术人员将能够在新的数据集上有效地测试和评估新的检索算法，而不会产生与标准信息检索评估范例相关的高成本。因此，该项目的一个方面是提供一种使能技术，它将促进新搜索算法的更快发展，这在当前的数字时代至关重要。从教育角度来看，该项目将培养研究生和本科生。

Search engines and other information retrieval technologies are critical in the digital age. The goal of the proposed research is to investigate novel frameworks for analyzing and efficiently evaluating measures of retrieval performance, with an eye toward fostering and enabling research leading to better search engines and other retrieval technologies. Two novel frameworks are proposed: (1) an information-theoretic framework within which one can quantifiably assess what various measures of retrieval performance are measuring and (2) a statistical framework within which one can efficiently estimate these measures of retrieval performance. The former provides a theoretical underpinning for retrieval evaluation and analysis; the latter provides a practical methodology for efficiently evaluating search engines on a large scale. Each will foster and enable research leading to better search engines and search technology. The potential impacts of this project are many. From a research and infrastructure perspective, the project will yield published research results and freely available software artifacts (via http://www.ccs.neu.edu/home/jaa/IIS-0534482/), which will permit large-scale retrieval evaluation with minimal effort. Academics and other technologists will be able to efficiently test and evaluate new retrieval algorithms on novel data sets without incurring the high costs associated with the standard information retrieval evaluation paradigm. As such, one aspect of the project is an enabling technology that will foster the more rapid development of new search algorithms, vital in the current digital age. From an educational perspective, the project will train graduate and undergraduate students.