CRI: CI-SUSTAIN: Collaborative Research: Sustaining Lemur Project Resources for the Long-Term

十多年来，由Lemur Project开发和提供的软件、数据集和在线服务支持并实现了大量关于搜索引擎、信息检索和其他分析和处理人类语言的计算机科学领域的学术和商业研究。该项目对Lemur项目的基础设施进行了重大改进，将基础设施再运营三年，并将其定位为长期可持续发展。作为增强功能的一部分，Galago搜索引擎得到了增强，以提供神经网络和其他机器学习方法的更强集成。开发了一个新的数据集，CNOW Web2020，以取代广泛使用的CNOW Web 09和CNOW Web 12数据集。这些投资将支持未来十年的先进研究。为该项目广泛用于研究的开放源码Indri和Galago搜索引擎开发的高级搜索功能被添加到工业界广泛使用的开放源码Lucene搜索引擎中。开发新的软件应用程序来简化Lemur Project搜索引擎和Lucene之间的迁移。这些投资提高了对行业至关重要的软件的最新水平，并使研究人员能够将研究转移到更广泛使用的软件上。狐猴项目的研究基础设施吸引了大量的研究用户社区，因为它可以轻松实现前沿研究。这些增强功能使信息检索和相关领域的研究人员能够进行更广泛的实验并分享他们的结果。由新的Lemur Project软件支持的研究和行业开发将为各种任务创建新一代更强大的搜索引擎。该项目围绕三种类型的活动组织：维护软件，维护数据集和操作。该项目通过增加对Indri和Galago功能的支持，并与开源Lucene搜索引擎创建集成和迁移路径，实现了软件的长期可持续性，该搜索引擎拥有庞大的用户和志愿者开发人员社区。因此，用Galago或Indri完成的研究将在Lucene中重现，并且更容易被Lucene的行业用户访问。该项目还扩展了Galago应用程序编程接口，以支持神经网络（深度学习）文档排名技术的最新发展，这些技术目前正在被广泛研究，并有望在最先进的研究系统中得到应用。它通过支持神经算法来更好地与高质量的学习排名方法进行比较，从而扩展了Ranklib的实用性，并通过支持其他文档和机器学习格式来扩展Sifaka文本挖掘应用程序的实用性。旧的CNOW Web 09和CNOW Web 12数据集被新的CNOW Web2020数据集取代，该数据集旨在持续十年，并支持对新的学习排名和神经网络（深度学习）排名算法的研究。该项目以软件维护和支持、数据集许可证发放和分发以及在线搜索服务运营的形式维护和运营现有基础设施。新的Lemur项目基础设施支持广泛的信息检索研究，例如，检索模型的研究;如何训练学习排名;使用半结构化知识库;结果多样化;查询优化;和分布式搜索。特别是，它大大提高了对学习和神经（深度学习）排名算法研究的支持，这些算法近年来已成为重要的研究课题。Cocktail Web数据集被广泛的人类语言技术研究社区使用。该项目使增强，维持这一基础设施的研究界至少在未来十年。这一奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的知识价值和更广泛的影响审查标准的支持。

For more than a decade, the software, datasets, and online services developed and provided by the Lemur Project have supported and enabled a large body of academic and commercial research on search engines, information retrieval, and other areas of computer science that analyze and process human language. This project makes critical enhancements to Lemur Project infrastructure, operates the infrastructure for another three years, and positions it for long-term sustainability. As part of the enhancements, the Galago search engine is enhanced to provide stronger integration of neural networks and other machine learning methods. A new dataset, ClueWeb2020, is developed to replace the widely-used ClueWeb09 and ClueWeb12 datasets. These investments will support advanced research for the next decade. The advanced search capabilities developed for the project's open-source Indri and Galago search engines, which are widely used for research, are added to the open-source Lucene search engine, which is widely used by industry. New software applications are developed to simplify migration between Lemur Project search engines and Lucene. These investments improve the state-of-the-art of software important to industry and enable researchers to migrate research to more widely-used software. The Lemur Project's research infrastructure attracted a substantial research user community because it easily enables leading-edge research. These enhancements enable researchers in information retrieval and related areas to carry out a much broader range of experiments and to share their results. Research and industry development supported by the new Lemur Project software will create a new generation of more capable search engines for a variety of tasks.The project is organized around three types of activities: Sustaining software, sustaining datasets, and operation. The project achieves long-term software sustainability by adding support for Indri and Galago functionality and creating integration and migration paths with the open-source Lucene search engine, which has large user and volunteer-developer communities. Research done with Galago or Indri will thus be reproducible in Lucene and more accessible to Lucene's industry users. The project also extends the Galago Application Programming Interface to support the newest developments in neural network (deep learning) document ranking technologies, which now are being studied widely and expected in a state-of-the-art research system. It broadens the utility of Ranklib by supporting neural algorithms for better comparison with high quality learning to rank approaches, and broadens the utility of the Sifaka text mining application with support for additional document and machine learning formats. The older ClueWeb09 and ClueWeb12 datsets are superseded by a new ClueWeb2020 dataset that is designed to last a decade and support research on newer learning-to-rank and neural network (deep learning) ranking algorithms. The project maintains and operates the existing infrastructure, in the form of software maintenance and support; dataset licensing and distribution; and operation of online search services. The new Lemur Project infrastructure supports a broad range of Information Retrieval research, for example, research on retrieval models; how to train learned rankers; use of semi-structured knowledge bases; result diversification; query optimization; and distributed search. In particular, it greatly improves support for research on learned and neural (deep learning) ranking algorithms, which have become important research topics in recent years. The ClueWeb datasets are used by a broad human language technologies research community. This project makes enhancements that sustain this infrastructure for the research community for at least the next decade.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.