Design Considerations for a Sustainable Scholarly Big Data Service
Design Considerations for a Sustainable Scholarly Big Data Service
复制标题
DOI：
10.1145/3574318.3574340
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Jian Wu;Shaurya Rohatgi;Manoj K. Angadi;Kavya S. Puranik;C. Lee Giles
中科院分区：
文献类型：
--
作者：
Jian Wu;Shaurya Rohatgi;Manoj K. Angadi;Kavya S. Puranik;C. Lee Giles
Web编程技术（如Ajax和jQuery）以及数据存储（如Apache Solr和Elasticsearch）的进步使得部署中小型基于Web的搜索引擎变得更加容易。然而，由于人力资源和财政支持有限，开发一个支持学术大数据服务的可持续搜索引擎仍然具有挑战性。这种情况在学术环境或小企业中很常见。在这里，我们展示了在开发下一代CiteSeerX（NGX）时，如何通过权衡性能，成本和效率等竞争因素来做出四个关键的设计决策，NGX是CiteSeerX的继任者，CiteSeerX是一个开创性的数字图书馆搜索引擎，已经为学术界服务了二十多年。这项工作扩展了我们之前在Wu等人（2021）中的工作，并讨论了基础设施，Web应用程序，索引和文档过滤的设计考虑因素。这些设计考虑可以推广到其他基于Web的搜索引擎，这些搜索引擎具有类似的规模，部署在资源有限的小型企业或学术机构中。
The advancement of web programming techniques, such as Ajax and jQuery, and datastores, such as Apache Solr and Elasticsearch, have made it much easier to deploy small to medium scale web-based search engines. However, developing a sustainable search engine that supports scholarly big data services is still challenging often because of limited human resources and financial support. Such scenarios are typical in academic settings or small businesses. Here, we showcase how four key design decisions were made by trading-off competing factors such as performance, cost, and efficiency, when developing the Next Generation CiteSeerX (NGX), the successor of CiteSeerX, which was a pioneering digital library search engine that has been serving academic communities for more than two decades. This work extends our previous work in Wu et al. (2021) and discusses design considerations of infrastructure, web applications, indexing, and document filtering. These design considerations can be generalized to other web-based search engines with a similar scale that are deployed in small business or academic settings with limited resources.