C-TEST: Supporting novelty and diversity in testfiles for search tuning
C-TEST: Supporting novelty and diversity in testfiles for search tuning
复制标题
C-TEST：支持测试文件中的新颖性和多样性以进行搜索调整
DOI：
--
复制
发表时间：
2009
期刊：
影响因子：
--
通讯作者：
Paul Thomas
中科院分区：
文献类型：
--
作者：
D. Hawking;Tom Rowlands;Paul Thomas
调整搜索机构，例如Web搜索引擎或在特定组织中部署的企业搜索工具，这是一项重要的活动，而不是在使用该设施的搜索者中最大程度地提高调整的最终目标。因此，调整应使用无偏见的实际搜索请求样本，一个对真实搜索者的限制过程，并衡量与实际满意度相关的过程必须反映出特定查询的结果可以由具有不同的潜在信息需求的提交者来判断。用于批量调整，但能够反映这些重要方面。有关某些TREC任务的关联工具包和C检验示例测试文件，可在es.csiro.au/c-test上找到。
Tuning a search facility such as a Web search engine, or an enterprise search tool deployed in a particular organisation, is an economically important activity. Intuitively, an important end goal of tuning should be to maximise satisfaction across the searchers who will use the facility. Tuning should therefore use an unbiased sample of actual search requests, a judging process accurately modelling that of real searchers, and measures optimally correlated with real satisfaction. The process must reflect that the result set for a particular query may be judged by submitters with different underlying information needs. It should also model users’ impatience with duplicate results. Here, we describe a formally-defined, practical, public-domain testfile format suitable for use in batch tuning but capable of reflecting these important aspects. We hope that the formats will soon be supported by all major academic IR systems. DTDs for test and result files, plus an associated toolkit and C-TEST example testfiles for some TREC tasks, are available at es.csiro.au/C-TEST.