Collaborative Research: Elements: FaaSr: Enabling Cloud-native Event-driven Function-as-a-Service Computing Workflows in R

该项目开发FaaSr，这是一种新的软件，将促进用R语言编写的科学计算应用程序在功能即服务（FaaS）云计算基础设施中的编程和部署。云计算的FaaS模型支持在自动供应和管理的服务器中动态、按需执行计算功能，其方式既经济高效又可扩展：用户不需要管理云服务器（包括按需扩展），也不需要为未使用的服务器的空闲时间付费。因此，FaaS模型在降低在云基础设施中执行科学计算的复杂性和成本方面具有很大的潜力。然而，到目前为止，FaaS平台主要被设计为支持基于web的应用程序，这导致了现有FaaS平台与科学界之间的主要差距。这种差距在环境科学领域尤其明显，因为R是环境科学领域的重点编程语言。这是因为：1)在FaaS平台中没有对R语言的原生支持，2)每个FaaS平台都有一个独特的接口来部署和管理由多个功能组成的工作流，从而为用户在一个或多个FaaS平台上开发和部署应用程序创造了障碍。该项目通过开发开源软件来加速科学应用程序中事件驱动的FaaS工作流的采用，从而弥合了这一差距。FaaSr软件将作为一个易于安装的R软件包发布，并将为程序员提供简单的接口，同时支持多种开源和商业云计算基础设施。该软件将支持广泛的科学计算应用，特别是那些需要动态事件驱动处理（如预测和连续数据质量）的环境科学子领域（包括生态学和生物多样性）。最终，该项目旨在开发可扩展、可推广和强大的工作流程，以提高生态预测的能力、实践和培训机会。生态预测是一个活跃的科学研究领域，有望显著提高有效环境决策和管理的预测能力。本项目开发的FaaS软件将极大地扩展FaaS云计算基础设施的采用。目前，在用R语言编写的科学应用程序能够充分发挥FaaS平台的潜力之前，还有许多重大挑战需要克服，因为R本身不受支持，而且不同的平台有不同的、不兼容的编程接口。此外，科学应用程序需要由动态执行的多个功能组成的工作流，并通过在云存储中以文件形式交换数据进行通信。不同的FaaS平台有不同的编程接口来实现这些功能，从而增加了开发人员和用户的复杂性。这个协作的、跨学科的项目通过集成分布式系统、生态学和预测方面的专业知识来设计和实现软件，克服了这些挑战，这些软件由科学计算用例驱动；创建易于使用的界面；并且建立在最先进的分布式计算技术和框架之上。具体来说，FaaSr软件将做出多种新颖的技术贡献，包括：1)它将允许最终用户在高抽象级别上使用R语言编写工作流；2)它将包括一个统一的、易于使用的接口，用于处理事件调用和参数解析，向开发人员隐藏了多个FaaS接口编程的复杂性，同时支持多个FaaS框架，包括GitHub Actions、OpenWhisk、IBM Cloud Functions和Amazon Web Services Lambda；3)它将包括一个易于使用的接口，用于处理云数据存储和访问，使用事实上的标准接口和文件格式隐藏底层细节（例如，访问端点和凭据）；4)它将实现一种统一的方法来组成有向无循环图工作流，可以自动映射到不同FaaS平台支持的编程接口。FaaSr的设计、实现和部署方面的经验将为分布式/云计算贡献新的技术和技术，作为该项目的一部分，湖泊和水库的研究为评估软件的性能、可扩展性和可用性提供了一个现实的测试平台。此外，我们的团队将建立并扩展我们现有的本科生和研究生跨学科研究交流项目，在计算机科学、淡水科学和生态系统建模的交叉领域提供新颖的培训。该奖项由NSF高级网络基础设施办公室颁发，并得到NSF生物科学理事会的联合支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project develops FaaSr, a new software that will facilitate the programming and deployment of scientific computing applications written in the R language in Function-as-a-Service (FaaS) cloud computing infrastructures. The FaaS model of cloud computing supports dynamic, on-demand execution of computing functions in servers that are automatically provisioned and managed, in a way that is both cost-effective and scalable: users do not need to manage cloud servers (including on-demand scaling) nor pay for idle time of unutilized servers. The FaaS model thus has much potential for reducing the complexity and cost of performing scientific computing in cloud infrastructures. To date, however, FaaS platforms have been primarily designed to support Web-based applications, resulting in a major gap between existing FaaS platforms and the scientific community. This gap is particularly evident in the environmental sciences, where R is the focal programming language. This is because: 1) there is no native support for the R language in FaaS platforms, and 2) each FaaS platform has a unique interface to deploy and manage workflows consisting of multiple functions, thereby creating barriers for users to develop and deploy applications on one or more FaaS platforms. This project bridges this gap by developing open-source software to accelerate the adoption of event-driven FaaS workflows for scientific applications. The FaaSr software will be distributed as an easy-to-install R package and will provide simple interfaces to programmers, while supporting multiple open-source and commercial cloud computing infrastructures. The software will support a wide range of scientific computing applications, in particular those that require dynamic event-driven processing (such as forecasting and continuous data quality) in environmental science subfields (including ecology and biodiversity). Ultimately, the project aims to develop scalable, generalizable, and robust workflows that will advance the capacity, practice, and training opportunities for ecological forecasting, an active area of scientific research poised to significantly increase predictive capacity for effective environmental decision-making and management. The FaaSr software developed in this project will greatly expand the adoption of FaaS cloud computing infrastructure. Currently, there are significant challenges to be overcome before scientific applications written in the R language can fully realize the potential of FaaS platforms, because R is not supported natively, and because different platforms have different, incompatible programming interfaces. Furthermore, scientific applications require workflows consisting of multiple functions that are executed dynamically and communicate by exchanging data as files in cloud storage. Different FaaS platforms have different programming interfaces to accomplish these capabilities, leading to increased complexity for developers and users. This collaborative, interdisciplinary project overcomes these challenges by integrating expertise in distributed systems, ecology, and forecasting together to design and implement software that: is driven by scientific computing use cases; creates easy-to-use interfaces; and builds on state-of-the-art distributed computing techniques and frameworks. Specifically, the FaaSr software will make multiple novel technical contributions, including: 1) it will allow end users to program a workflow at a high abstraction level and with the R language; 2) it will include a unified, easy-to-use interface for handling event invocation and argument parsing that hides the complexity of programming for multiple FaaS interfaces from developers, while supporting multiple FaaS frameworks, including GitHub Actions, OpenWhisk, IBM Cloud Functions, and Amazon Web Services Lambda; 3) it will include an easy-to-use interface for handling cloud data storage and access that hides low-level details (e.g., access endpoints and credentials) using de-facto standard interfaces and file formats; and 4) it will implement a unified approach to compose directed acyclic graph workflows that can be automatically mapped to programming interfaces supported by different FaaS platforms. Experiences with the design, implementation, and deployment of FaaSr will contribute new techniques and technologies in distributed/cloud computing, with lakes and reservoirs studied as part of this project providing a realistic testbed for assessing performance, extensibility, and availability of the software. Furthermore, our team will build on and expand our existing program for cross-disciplinary research exchanges of undergraduate and graduate students that provide novel training at the intersection of computer science, freshwater science, and ecosystem modeling.This award by the NSF Office of Advanced Cyberinfrastructure is jointly supported by the NSF Directorate for Biological Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.