CI-ADDO-EN: Collaborative Proposal: Supporting Web-Scale Experimentation Using the Lemur Toolkit

该项目维护和加强现有的社区软件基础设施，并创建新的社区数据基础设施，使信息检索研究界和相关研究界能够进行“网络规模”的研究，这意味着十亿或更多网页的数据集以及大量查询日志。软件基础设施以Lemur工具包和相关的Indri搜索引擎为基础，由于支持多种检索模型、多种形式的证据和强大的概率查询语言，许多信息检索研究人员使用这些工具。Lemur的增强包括支持流行的MapReduce风格的分布式处理和其他效率改进，使在常见的计算机硬件环境中进行大型Web数据集的研究变得实用。新的数据基础设施包括维护和分发一个新创建的十亿页数据集、另一个新的网络数据集以及与这些数据集匹配的大型匿名搜索日志。大型数据集和相应的大型搜索日志的组合使广泛的社区能够使用比以前更现实的数据资源进行研究。这项研究将进一步促进对大规模个性化搜索潜在问题的理解，这将是下一代搜索引擎的重要组成部分。欲了解更多信息，请访问项目网站：http://www.lemurproject.org.

This project maintains and enhances existing community software infrastructure, and creates new community data infrastructure to enable the information retrieval research community and related research communities to conduct research on a "web scale", meaning datasets of a billion or more web pages together with large query logs. The software infrastructure is based on the Lemur Toolkit and the associated Indri search engine, which are used by many information retrieval researchers due to the support for multiple retrieval models, multiple forms of evidence, and a powerful probabilistic query language. The enhancements to Lemur include support for the popular MapReduce style of distributed processing and other efficiency improvements to make it practical to do research on large web datasets 'out of the box' in common computer hardware environments. The new data infrastructure consists of maintenance and distribution of a newly created billion-page dataset, another new web dataset, and large, anonymized search logs that match the datasets. The combination of large datasets and corresponding large search logs enable a broad community to conduct research with more realistic data resources than were available previously. This research will lead to further advances in the understanding of the underlying issues for large-scale, personalized search, which will be an important part of the next generation of search engines. For further information, see the project web site at the URL: http://www.lemurproject.org.