Collaborative Research: Frameworks: funcX: A Function Execution Service for Portability and Performance

协作研究：框架：funcX：可移植性和性能的函数执行服务

• 批准号: 2004894
• 负责人: Ian Foster
• 金额: $ 265.81万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-05-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2004894&HistoricalAwards=false
• 关键词: Collaborative; Research; Frameworks; funcX; Function

The funcX project is developing, deploying, and operating a new distributed computing cyberinfrastructure platform to enable researchers to build applications from programming functions that execute on different computing resources, from laptops to supercomputers. This cloud-hosted service democratizes access to advanced computing by providing intuitive interfaces for both registering remote computers as function executors and executing functions on these computers reliably, securely, and with high performance. Researchers can thus decompose monolithic applications into collections of reusable lightweight functions that can be run wherever makes the most sense, for example where data reside or where excess capacity is available. By simplifying access to specialized and high performance cyberinfrastructure and decreasing the time to discovery, the project serves the national interest, as stated in NSF's mission, by promoting the progress of science. A total of 33 diverse science, cyberinfrastructure, and software institute partners working with cutting-edge science applications and research cyberinfrastructure will directly benefit from the funcX platform.This project develops funcX, a scalable and high-performance federated platform for managing the remote execution of (often short-duration) functions across diverse cyberinfrastructure systems, from edge accelerators to clusters, supercomputers, and clouds. funcX allows developers to decompose applications into collections of functions that can each be executed in the best location, in terms of cost, execution time, data movement costs, and/or energy consumption. It thus integrates the extreme convenience of the function as a service (FaaS) model, developed in industry for specific industry applications, with support for the specialized needs of scientific research. funcX addresses important barriers to these new uses of research cyberinfrastructure systems, by enabling the intuitive, flexible, and scalable execution of functions without regard to physical location, scheduler architecture, virtualization technology, administrative domain, or data location. Flexible open-source funcX agent software makes it easy to expose arbitrary computing systems as funcX computing platforms, thereby transforming existing cyberinfrastructure systems into high-performance function serving environments (endpoints). The cloud-hosted funcX service provides a REST interface for registering functions, discovering available endpoints, and managing the execution of functions on endpoints, all via a universal trust fabric and standard web authentication and authorization mechanisms. It dynamically creates and deploys containers that incorporate function dependencies and provide a secure and isolated environment for safe function execution. The project engages a diverse set of 11 science partners, 18 research computing and cyberinfrastructure projects, and 4 NSF Software Institutes, each supporting many NSF-funded researchers, to provide use cases for funcX, shape its design, and evaluate its implementation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Funcx项目正在开发，部署和操作一个新的分布式计算网络基础架构平台，以使研究人员能够从在不同计算资源上执行的编程函数构建应用程序，从笔记本电脑到超级计算机。这项云托管服务通过为远程计算机作为功能执行者提供直观的接口来使对高级计算的访问民主化，并在这些计算机上可靠，安全且具有高性能。因此，研究人员可以将整体应用分解为可重复使用的轻质功能的集合，这些功能可以在最有意义的地方运行，例如数据驻留的位置或可用的容量过剩。通过简化获得专业和高性能网络基础设施的访问，并减少了发现的时间，该项目通过促进科学的进步来实现国家利益，如NSF的使命中所述。 A total of 33 diverse science, cyberinfrastructure, and software institute partners working with cutting-edge science applications and research cyberinfrastructure will directly benefit from the funcX platform.This project develops funcX, a scalable and high-performance federated platform for managing the remote execution of (often short-duration) functions across diverse cyberinfrastructure systems, from edge accelerators to clusters, supercomputers, and云。 FunCX允许开发人员将应用程序分解为可以在成本，执行时间，数据移动成本和/或能源消耗的最佳位置执行的功能集合中。因此，它整合了该功能作为服务（FAAS）模型的极端便利，该模型在行业中针对特定行业应用开发，并支持科学研究的专业需求。 Funcx通过在不考虑物理位置，调度程序架构，虚拟化技术，管理领域或数据位置的情况下实现直观，灵活和可扩展的功能执行，解决了Cyber​​infrastructure Systems的这些新用途的重要障碍。灵活的开源Funcx代理软件使将任意计算系统作为funcx计算平台变得容易，从而将现有的Cyber​​inFrastructure Systems转换为高性能功能服务环境（端点）。云托管的Funcx服务提供了一个用于注册功能，发现可用端点并管理端点上的函数执行的REST接口，这都是通过通用信任结构和标准的Web身份验证和授权机制。它动态创建并部署了包含功能依赖性的容器，并为安全功能执行提供了安全且孤立的环境。该项目与11个科学合作伙伴，18个研究计算和网络基础设施项目以及4家NSF软件机构的各种各样的参与，每个机构都支持许多NSF资助的研究人员，以提供用于Funcx，塑造其设计并评估其实施的用例，并评估其奖项。这些奖项反映了NSF的法定任务，并通过评估范围来弥补企业的构成群体的范围，并构成了构成的范围。

项目成果

Enhancing Automated FaaS with Cost-aware Provisioning of Cloud Resources

通过具有成本意识的云资源配置来增强自动化 FaaS

• DOI: 10.1109/escience51609.2021.00053
• 发表时间: 2021
• 期刊: 2021 IEEE 17th International Conference on eScience (eScience
• 作者: Baughman, Matt;Foster, Ian;Chard, Kyle
• 通讯作者: Chard, Kyle

? unc X: Federated Function as a Service for Science

？

• DOI: 10.1109/tpds.2022.3208767
• 发表时间: 2022
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Li, Zhuozhao;Chard, Ryan;Babuji, Yadu;Galewsky, Ben;Skluzacek, Tyler J.;Nagaitsev, Kirill;Woodard, Anna;Blaiszik, Ben;Bryan, Josh;Katz, Daniel S.
• 通讯作者: Katz, Daniel S.

A Serverless Framework for Distributed Bulk Metadata Extraction

用于分布式批量元数据提取的无服务器框架

• DOI: 10.1145/3431379.3460636
• 发表时间: 2021
• 期刊: Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing (HPDC
• 作者: Skluzacek, Tyler J.;Wong, Ryan;Li, Zhuozhao;Chard, Ryan;Chard, Kyle;Foster, Ian
• 通讯作者: Foster, Ian

DLHub: Simplifying publication, discovery, and use of machine learning models in science

• DOI: 10.1016/j.jpdc.2020.08.006
• 发表时间: 2021-01-01
• 期刊: JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING
• 影响因子: 3.8
• 作者: Li, Zhuozhao;Chard, Ryan;Foster, Ian
• 通讯作者: Foster, Ian

funcX: a federated function serving fabric for science

• DOI: 10.1145/3369583.3392683
• 发表时间: 2020-06-23
• 期刊: HPDC '20: Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Chard, Ryan;Babuji, Yadu;Chard, Kyle
• 通讯作者: Chard, Kyle