Collaborative Research: Elements: FaaSr: Enabling Cloud-native Event-driven Function-as-a-Service Computing Workflows in R

协作研究：要素：FaaSr：在 R 中启用云原生事件驱动的函数即服务计算工作流程

• 批准号: 2311123
• 负责人: Renato Figueiredo
• 金额: $ 50万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-15 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311123&HistoricalAwards=false
• 关键词: Collaborative; Research; Elements; FaaSr; Enabling

This project develops FaaSr, a new software that will facilitate the programming and deployment of scientific computing applications written in the R language in Function-as-a-Service (FaaS) cloud computing infrastructures. The FaaS model of cloud computing supports dynamic, on-demand execution of computing functions in servers that are automatically provisioned and managed, in a way that is both cost-effective and scalable: users do not need to manage cloud servers (including on-demand scaling) nor pay for idle time of unutilized servers. The FaaS model thus has much potential for reducing the complexity and cost of performing scientific computing in cloud infrastructures. To date, however, FaaS platforms have been primarily designed to support Web-based applications, resulting in a major gap between existing FaaS platforms and the scientific community. This gap is particularly evident in the environmental sciences, where R is the focal programming language. This is because: 1) there is no native support for the R language in FaaS platforms, and 2) each FaaS platform has a unique interface to deploy and manage workflows consisting of multiple functions, thereby creating barriers for users to develop and deploy applications on one or more FaaS platforms. This project bridges this gap by developing open-source software to accelerate the adoption of event-driven FaaS workflows for scientific applications. The FaaSr software will be distributed as an easy-to-install R package and will provide simple interfaces to programmers, while supporting multiple open-source and commercial cloud computing infrastructures. The software will support a wide range of scientific computing applications, in particular those that require dynamic event-driven processing (such as forecasting and continuous data quality) in environmental science subfields (including ecology and biodiversity). Ultimately, the project aims to develop scalable, generalizable, and robust workflows that will advance the capacity, practice, and training opportunities for ecological forecasting, an active area of scientific research poised to significantly increase predictive capacity for effective environmental decision-making and management. The FaaSr software developed in this project will greatly expand the adoption of FaaS cloud computing infrastructure. Currently, there are significant challenges to be overcome before scientific applications written in the R language can fully realize the potential of FaaS platforms, because R is not supported natively, and because different platforms have different, incompatible programming interfaces. Furthermore, scientific applications require workflows consisting of multiple functions that are executed dynamically and communicate by exchanging data as files in cloud storage. Different FaaS platforms have different programming interfaces to accomplish these capabilities, leading to increased complexity for developers and users. This collaborative, interdisciplinary project overcomes these challenges by integrating expertise in distributed systems, ecology, and forecasting together to design and implement software that: is driven by scientific computing use cases; creates easy-to-use interfaces; and builds on state-of-the-art distributed computing techniques and frameworks. Specifically, the FaaSr software will make multiple novel technical contributions, including: 1) it will allow end users to program a workflow at a high abstraction level and with the R language; 2) it will include a unified, easy-to-use interface for handling event invocation and argument parsing that hides the complexity of programming for multiple FaaS interfaces from developers, while supporting multiple FaaS frameworks, including GitHub Actions, OpenWhisk, IBM Cloud Functions, and Amazon Web Services Lambda; 3) it will include an easy-to-use interface for handling cloud data storage and access that hides low-level details (e.g., access endpoints and credentials) using de-facto standard interfaces and file formats; and 4) it will implement a unified approach to compose directed acyclic graph workflows that can be automatically mapped to programming interfaces supported by different FaaS platforms. Experiences with the design, implementation, and deployment of FaaSr will contribute new techniques and technologies in distributed/cloud computing, with lakes and reservoirs studied as part of this project providing a realistic testbed for assessing performance, extensibility, and availability of the software. Furthermore, the team will build on and expand its existing program for cross-disciplinary research exchanges of undergraduate and graduate students that provide novel training at the intersection of computer science, freshwater science, and ecosystem modeling.This award by the NSF Office of Advanced Cyberinfrastructure is jointly supported by the NSF Directorate for Biological Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目开发了Faasr，这是一个新的软件，将促进在功能即服务(FAAS)云计算基础设施中用R语言编写的科学计算应用程序的编程和部署。云计算的FAAS模型支持在自动配置和管理的服务器中以经济高效且可扩展的方式动态、按需执行计算功能：用户不需要管理云服务器(包括按需扩展)，也不需要为未使用的服务器的空闲时间付费。因此，FAAS模型在降低在云基础设施中执行科学计算的复杂性和成本方面具有很大潜力。然而，到目前为止，FAAS平台主要是为支持基于Web的应用程序而设计的，导致现有FAAS平台与科学界之间存在很大差距。这一差距在环境科学中尤为明显，R是重点编程语言。这是因为：1)FAAS平台没有对R语言的本地支持，2)每个FAAS平台都有一个独特的接口来部署和管理由多种功能组成的工作流，从而为用户在一个或多个FAAS平台上开发和部署应用程序制造了障碍。该项目通过开发开源软件来加快科学应用程序采用事件驱动的FAAS工作流程，从而弥合了这一差距。FAASR软件将以易于安装的R包的形式分发，并将为程序员提供简单的界面，同时支持多种开源和商业云计算基础设施。该软件将支持广泛的科学计算应用，特别是那些在环境科学分领域(包括生态学和生物多样性)需要动态事件驱动处理(如预测和连续数据质量)的应用。最终，该项目旨在开发可扩展、可推广和强大的工作流程，以提高生态预测的能力、实践和培训机会，这是一个活跃的科学研究领域，有望显著提高有效环境决策和管理的预测能力。本项目开发的FAASR软件将极大地扩大FAAS云计算基础设施的采用。目前，在用R语言编写的科学应用程序能够充分实现FAAS平台的潜力之前，有许多重大挑战需要克服，因为R本身并不受支持，而且不同的平台具有不同的、不兼容的编程接口。此外，科学应用需要由多个功能组成的工作流，这些功能可以动态执行，并通过在云存储中以文件形式交换数据来进行通信。不同的FAAS平台具有不同的编程接口来实现这些功能，导致开发人员和用户的复杂性增加。这个协作的跨学科项目通过将分布式系统、生态和预测方面的专业知识整合在一起来设计和实施软件，从而克服了这些挑战：由科学计算用例驱动；创建易于使用的界面；并基于最先进的分布式计算技术和框架。具体地说，FaaSR软件将做出多项新的技术贡献，包括：1)它将允许最终用户在高抽象级别上使用R语言编写工作流；2)它将包括一个统一的、易于使用的接口，用于处理事件调用和参数解析，向开发人员隐藏了多个FAAS接口的编程复杂性，同时支持多个FAAS框架，包括GitHub Actions、OpenWhisk、IBM Cloud Functions和Amazon Web Services Lambda；3)它将包括一个易于使用的接口，用于处理云数据存储和访问，该接口使用事实上的标准接口和文件格式隐藏低层细节(例如，访问端点和凭据)；4)它将实施统一的方法来组成可自动映射到不同FAAS平台支持的编程接口的有向非循环图工作流。FAaSR的设计、实施和部署经验将有助于分布式/云计算领域的新技术和新技术，将湖泊和水库作为该项目的一部分进行研究，为评估该软件的性能、可扩展性和可用性提供了一个现实的试验台。此外，该团队将在现有的本科生和研究生跨学科研究交流计划的基础上，扩大现有计划，在计算机科学、淡水科学和生态系统建模的交叉点提供新的培训。该奖项由NSF高级网络基础设施办公室联合支持，由NSF生物科学局共同支持。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。