Pooling for a Large-Scale Test Collection: An Analysis of the Search Results from the First NTCIR Workshop
Pooling for a Large-Scale Test Collection: An Analysis of the Search Results from the First NTCIR Workshop
复制标题
大规模测试集合的汇集：对第一届 NTCIR 研讨会搜索结果的分析
DOI：
10.1023/a:1012778807438
复制
发表时间：
2004
期刊：
影响因子：
--
通讯作者：
K. Eguchi
中科院分区：
文献类型：
--
作者：
K. Kuriyama;N. Kando;T. Nozue;K. Eguchi
我们进行了一项研究：（1）验证池的详尽性，以构建大规模测试集合，以及（2）检查池文档数量的差异是否会影响IR系统的相对评估。我们使用第一届NTCIR研讨会的预测试和测试中提交的搜索主题及其相关性评估和搜索结果进行了实验，我们的结果验证了池化方法的效率和有效性，相关性评估的详尽性，以及基于池化方法的测试集评估的可靠性。
We have conducted a study to: (1) verify the exhaustiveness of pooling for the purpose of constructing a large-scale test collection, and (2) examine whether a difference in the number of pool documents can affect the relative evaluation of IR systems. We carried out the experiments using search topics, their relevance assessments, and the search results that were submitted for both the pre-test and test of the first NTCIR Workshop.Our results verified the efficiency and the effectiveness of the pooling method, the exhaustiveness of the relevance assessments, and the reliability of the evaluation using the test collection based on the pooling method.