CRI: CI-SUSTAIN: Collaborative Research: Sustaining Lemur Project Resources for the Long-Term

十多年来，Lemur项目开发和提供的软件、数据集和在线服务支持了大量关于搜索引擎、信息检索和其他分析和处理人类语言的计算机科学领域的学术和商业研究，并使之成为可能。该项目对狐猴项目的基础设施进行了重大改进，使基础设施再运行三年，并使其具有长期可持续性。作为增强的一部分，Galago搜索引擎得到了增强，以提供更强的神经网络和其他机器学习方法的集成。开发了一个新的数据集ClueWeb2020，以取代广泛使用的ClueWeb09和ClueWeb12数据集。这些投资将支持未来十年的先进研究。为该项目的开源Indri和Galago搜索引擎开发的高级搜索功能被添加到广泛用于研究的开源Lucene搜索引擎中。开发了新的软件应用程序来简化Lemur项目搜索引擎和Lucene之间的迁移。这些投资提高了对工业很重要的软件的技术水平，并使研究人员能够将研究转移到更广泛使用的软件上。Lemur项目的研究基础设施吸引了大量的研究用户社区，因为它很容易实现前沿研究。这些改进使信息检索和相关领域的研究人员能够开展更广泛的实验并分享他们的结果。新的Lemur项目软件支持的研究和行业发展将为各种任务创造新一代更强大的搜索引擎。该项目围绕三种类型的活动进行组织：维护软件、维护数据集和运营。该项目通过增加对Indri和Galago功能的支持，并与开源的Lucene搜索引擎创建集成和迁移路径，实现了长期的软件可持续性，Lucene搜索引擎拥有大量用户和志愿者开发人员社区。因此，在Galago或Indri上完成的研究可以在Lucene中重现，并且更容易被Lucene的行业用户访问。该项目还扩展了Galago应用程序编程接口，以支持神经网络（深度学习）文档排序技术的最新发展，该技术目前正在广泛研究，并有望在最先进的研究系统中得到应用。它通过支持神经算法来更好地与高质量的学习排序方法进行比较，从而扩大了Ranklib的实用性，并且通过支持额外的文档和机器学习格式，扩大了Sifaka文本挖掘应用程序的实用性。旧的ClueWeb09和ClueWeb12数据集被新的ClueWeb2020数据集所取代，该数据集旨在持续十年，并支持更新的学习排名和神经网络（深度学习）排名算法的研究。该项目以软件维护和支持的形式维护和操作现有的基础设施；数据集许可和分发；以及在线搜索服务的运营。新的狐猴项目基础设施支持广泛的信息检索研究，例如检索模型研究；如何训练有经验的排名员；半结构化知识库的使用；结果多样化;查询优化;分布式搜索。特别是，它大大提高了对学习和神经（深度学习）排序算法的研究支持，这些算法近年来已成为重要的研究课题。ClueWeb数据集被广泛的人类语言技术研究社区所使用。该项目对基础设施进行了改进，使其至少在未来十年内为研究界提供支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

For more than a decade, the software, datasets, and online services developed and provided by the Lemur Project have supported and enabled a large body of academic and commercial research on search engines, information retrieval, and other areas of computer science that analyze and process human language. This project makes critical enhancements to Lemur Project infrastructure, operates the infrastructure for another three years, and positions it for long-term sustainability. As part of the enhancements, the Galago search engine is enhanced to provide stronger integration of neural networks and other machine learning methods. A new dataset, ClueWeb2020, is developed to replace the widely-used ClueWeb09 and ClueWeb12 datasets. These investments will support advanced research for the next decade. The advanced search capabilities developed for the project's open-source Indri and Galago search engines, which are widely used for research, are added to the open-source Lucene search engine, which is widely used by industry. New software applications are developed to simplify migration between Lemur Project search engines and Lucene. These investments improve the state-of-the-art of software important to industry and enable researchers to migrate research to more widely-used software. The Lemur Project's research infrastructure attracted a substantial research user community because it easily enables leading-edge research. These enhancements enable researchers in information retrieval and related areas to carry out a much broader range of experiments and to share their results. Research and industry development supported by the new Lemur Project software will create a new generation of more capable search engines for a variety of tasks.The project is organized around three types of activities: Sustaining software, sustaining datasets, and operation. The project achieves long-term software sustainability by adding support for Indri and Galago functionality and creating integration and migration paths with the open-source Lucene search engine, which has large user and volunteer-developer communities. Research done with Galago or Indri will thus be reproducible in Lucene and more accessible to Lucene's industry users. The project also extends the Galago Application Programming Interface to support the newest developments in neural network (deep learning) document ranking technologies, which now are being studied widely and expected in a state-of-the-art research system. It broadens the utility of Ranklib by supporting neural algorithms for better comparison with high quality learning to rank approaches, and broadens the utility of the Sifaka text mining application with support for additional document and machine learning formats. The older ClueWeb09 and ClueWeb12 datsets are superseded by a new ClueWeb2020 dataset that is designed to last a decade and support research on newer learning-to-rank and neural network (deep learning) ranking algorithms. The project maintains and operates the existing infrastructure, in the form of software maintenance and support; dataset licensing and distribution; and operation of online search services. The new Lemur Project infrastructure supports a broad range of Information Retrieval research, for example, research on retrieval models; how to train learned rankers; use of semi-structured knowledge bases; result diversification; query optimization; and distributed search. In particular, it greatly improves support for research on learned and neural (deep learning) ranking algorithms, which have become important research topics in recent years. The ClueWeb datasets are used by a broad human language technologies research community. This project makes enhancements that sustain this infrastructure for the research community for at least the next decade.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.