CI-ADDO-EN: Collaborative Proposal: Supporting Web-Scale Experimentation using the Lemur Toolkit

该项目维护和加强现有的社区软件基础设施，并创建新的社区数据基础设施，使信息检索研究社区和相关研究社区能够进行“网络规模”的研究，这意味着十亿或更多网页的数据集以及大量查询日志。软件基础设施基于Lemur工具包和相关的Indri搜索引擎，由于支持多种检索模型、多种形式的证据和强大的概率查询语言，许多信息检索研究人员使用该工具。对Lemur的增强包括支持流行的MapReduce型分布式处理和其他效率改进，以使在普通计算机硬件环境中进行大型Web数据集的研究变得实用。新的数据基础设施包括维护和分发新创建的10亿页数据集、另一个新的Web数据集，以及与数据集匹配的大型匿名搜索日志。大型数据集和相应的大型搜索日志相结合，使更广泛的社区能够使用比以前更现实的数据资源进行研究。这项研究将进一步促进对大规模个性化搜索基本问题的理解，这将是下一代搜索引擎的重要组成部分。有关更多信息，请参阅该项目的网站：http://www.lemurproject.org.

This project maintains and enhances existing community softwareinfrastructure, and creates new community data infrastructure toenable the information retrieval research community and relatedresearch communities to conduct research on a "web scale",meaning datasets of a billion or more web pages together withlarge query logs. The software infrastructure is based on theLemur Toolkit and the associated Indri search engine, which areused by many information retrieval researchers due to the supportfor multiple retrieval models, multiple forms of evidence, and apowerful probabilistic query language. The enhancements to Lemurinclude support for the popular MapReduce style of distributedprocessing and other efficiency improvements to make it practicalto do research on large web datasets 'out of the box' in commoncomputer hardware environments.The new data infrastructure consists of maintenance anddistribution of a newly created billion-page dataset, another newweb dataset, and large, anonymized search logs that match thedatasets. The combination of large datasets and correspondinglarge search logs enable a broad community to conduct researchwith more realistic data resources than were availablepreviously. This research will lead to further advances in theunderstanding of the underlying issues for large-scale,personalized search, which will be an important part of the nextgeneration of search engines.For further information, see the project web site at the URL:http://www.lemurproject.org.