Collaborative Research: Elements: FaaSr: Enabling Cloud-native Event-driven Function-as-a-Service Computing Workflows in R

该项目开发FaaSr，这是一种新的软件，将促进在功能即服务（FaaS）云计算基础设施中以R语言编写的科学计算应用程序的编程和部署。云计算的FaaS模型支持在自动配置和管理的服务器中动态按需执行计算功能，其方式既具有成本效益又具有可扩展性：用户无需管理云服务器（包括按需扩展），也无需为未使用的服务器的空闲时间付费。因此，FaaS模型在降低云基础设施中执行科学计算的复杂性和成本方面具有很大的潜力。然而，到目前为止，FaaS平台主要设计用于支持基于Web的应用程序，导致现有FaaS平台与科学界之间存在重大差距。这种差距在环境科学中尤其明显，R是重点编程语言。这是因为：1）FaaS平台中没有对R语言的原生支持，2）每个FaaS平台都有唯一的接口来部署和管理由多个功能组成的工作流，从而为用户在一个或多个FaaS平台上开发和部署应用程序创造了障碍。该项目通过开发开源软件来弥合这一差距，以加速科学应用程序采用事件驱动的FaaS工作流。FaaSr软件将作为易于安装的R包分发，并将为程序员提供简单的界面，同时支持多种开源和商业云计算基础设施。该软件将支持广泛的科学计算应用，特别是在环境科学分领域（包括生态学和生物多样性）中需要动态事件驱动处理的应用（如预测和连续数据质量）。最终，该项目旨在开发可扩展，可推广和强大的工作流程，以提高生态预测的能力，实践和培训机会，这是一个活跃的科学研究领域，有望显着提高有效环境决策和管理的预测能力。该项目中开发的FaaSr软件将大大扩展FaaS云计算基础设施的采用。目前，在用R语言编写的科学应用程序能够完全实现FaaS平台的潜力之前，还有一些重大挑战需要克服，因为R本身并不支持，而且不同的平台具有不同的、不兼容的编程接口。此外，科学应用程序需要由多个功能组成的工作流，这些功能可以动态执行，并通过在云存储中交换数据作为文件进行通信。不同的FaaS平台具有不同的编程接口来实现这些功能，这导致开发人员和用户的复杂性增加。这个跨学科的协作项目通过将分布式系统，生态学和预测方面的专业知识整合在一起来克服这些挑战，以设计和实施软件：由科学计算用例驱动;创建易于使用的界面;并建立在最先进的分布式计算技术和框架之上。具体来说，FaaSr软件将做出多项新的技术贡献，包括：1）它将允许最终用户在高抽象级别上使用R语言编程工作流程; 2）它将包括一个统一的、易于使用的接口，用于处理事件调用和参数解析，向开发人员隐藏了多个FaaS接口编程的复杂性，同时支持多个FaaS框架，包括GitHub Actions、OpenWhisk、IBM Cloud Functions和Amazon Web Services Lambda; 3）它将包括一个易于使用的界面，用于处理隐藏低级细节的云数据存储和访问（例如，访问端点和凭证）;以及4）它将实现统一的方法来组成有向无环图工作流，这些工作流可以自动映射到不同FaaS平台支持的编程接口。FaaSr的设计，实施和部署经验将有助于分布式/云计算的新技术和新技术，作为该项目的一部分，湖泊和水库的研究提供了一个现实的测试平台，用于评估软件的性能，可扩展性和可用性。此外，我们的团队将建立和扩大我们现有的本科生和研究生跨学科研究交流计划，在计算机科学，淡水科学，和生态系统建模。该奖项由NSF高级网络基础设施办公室颁发，并得到NSF生物科学理事会的共同支持。该奖项反映了NSF的法定使命，并被认为是值得的。通过使用基金会的知识价值和更广泛的影响审查标准进行评估来提供支持。

This project develops FaaSr, a new software that will facilitate the programming and deployment of scientific computing applications written in the R language in Function-as-a-Service (FaaS) cloud computing infrastructures. The FaaS model of cloud computing supports dynamic, on-demand execution of computing functions in servers that are automatically provisioned and managed, in a way that is both cost-effective and scalable: users do not need to manage cloud servers (including on-demand scaling) nor pay for idle time of unutilized servers. The FaaS model thus has much potential for reducing the complexity and cost of performing scientific computing in cloud infrastructures. To date, however, FaaS platforms have been primarily designed to support Web-based applications, resulting in a major gap between existing FaaS platforms and the scientific community. This gap is particularly evident in the environmental sciences, where R is the focal programming language. This is because: 1) there is no native support for the R language in FaaS platforms, and 2) each FaaS platform has a unique interface to deploy and manage workflows consisting of multiple functions, thereby creating barriers for users to develop and deploy applications on one or more FaaS platforms. This project bridges this gap by developing open-source software to accelerate the adoption of event-driven FaaS workflows for scientific applications. The FaaSr software will be distributed as an easy-to-install R package and will provide simple interfaces to programmers, while supporting multiple open-source and commercial cloud computing infrastructures. The software will support a wide range of scientific computing applications, in particular those that require dynamic event-driven processing (such as forecasting and continuous data quality) in environmental science subfields (including ecology and biodiversity). Ultimately, the project aims to develop scalable, generalizable, and robust workflows that will advance the capacity, practice, and training opportunities for ecological forecasting, an active area of scientific research poised to significantly increase predictive capacity for effective environmental decision-making and management. The FaaSr software developed in this project will greatly expand the adoption of FaaS cloud computing infrastructure. Currently, there are significant challenges to be overcome before scientific applications written in the R language can fully realize the potential of FaaS platforms, because R is not supported natively, and because different platforms have different, incompatible programming interfaces. Furthermore, scientific applications require workflows consisting of multiple functions that are executed dynamically and communicate by exchanging data as files in cloud storage. Different FaaS platforms have different programming interfaces to accomplish these capabilities, leading to increased complexity for developers and users. This collaborative, interdisciplinary project overcomes these challenges by integrating expertise in distributed systems, ecology, and forecasting together to design and implement software that: is driven by scientific computing use cases; creates easy-to-use interfaces; and builds on state-of-the-art distributed computing techniques and frameworks. Specifically, the FaaSr software will make multiple novel technical contributions, including: 1) it will allow end users to program a workflow at a high abstraction level and with the R language; 2) it will include a unified, easy-to-use interface for handling event invocation and argument parsing that hides the complexity of programming for multiple FaaS interfaces from developers, while supporting multiple FaaS frameworks, including GitHub Actions, OpenWhisk, IBM Cloud Functions, and Amazon Web Services Lambda; 3) it will include an easy-to-use interface for handling cloud data storage and access that hides low-level details (e.g., access endpoints and credentials) using de-facto standard interfaces and file formats; and 4) it will implement a unified approach to compose directed acyclic graph workflows that can be automatically mapped to programming interfaces supported by different FaaS platforms. Experiences with the design, implementation, and deployment of FaaSr will contribute new techniques and technologies in distributed/cloud computing, with lakes and reservoirs studied as part of this project providing a realistic testbed for assessing performance, extensibility, and availability of the software. Furthermore, our team will build on and expand our existing program for cross-disciplinary research exchanges of undergraduate and graduate students that provide novel training at the intersection of computer science, freshwater science, and ecosystem modeling.This award by the NSF Office of Advanced Cyberinfrastructure is jointly supported by the NSF Directorate for Biological Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.