CRII: OAC: High-Efficiency Serverless Computing Systems for Deep Learning: A Hybrid CPU/GPU Architecture

CRII：OAC：用于深度学习的高效无服务器计算系统：混合 CPU/GPU 架构

• 批准号: 2153502
• 负责人: Hao Wang
• 金额: $ 17.49万
• 依托单位: Louisiana State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-05-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2153502&HistoricalAwards=false
• 关键词: CRII; OAC; Efficiency; Serverless; Computing

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Next-generation serverless cloud computing provides developers with simplified access to server management and administration, including event-driven execution, fine-grained resource provisioning, auto-scaling, and pay-as-you-go billing. The machine learning community is taking advantage of these benefits of serverless cloud computing to ease the development and deployment of deep learning (DL) applications. However, existing serverless computing platforms lack efficient support for GPUs, impeding DL practitioners from utilizing serverless computing for large-scale applications. This project will develop an efficient serverless computing platform with a hybrid CPU/GPU architecture to accelerate DL application development and deployment. The goal is to advance cutting-edge methodologies in both deep learning and serverless computing, which will result in a significant leap forward to benefit DL practitioners, DL users, and providers of cloud computing infrastructures, contributing to science advancement for society. The research findings will also enhance undergraduate and graduate education with exciting examples and demonstrations of real-world systems at the intersection of distributed computing, cloud computing, and deep learning.The project will develop a novel serverless computing platform with a hybrid CPU/GPU architecture that will provide DL applications with native GPU performance. Two core components constitute the hybrid serverless computing architecture, a shim virtualized GPU (vGPU) layer and a refactored container subsystem. The shim vGPU layer enables high-performance GPU sharing for concurrent serverless functions with low latency and high scalability. This layer provides fine-grained GPU resource provisioning and performance isolation by intercepting GPU calls from serverless functions using API remoting techniques. The vGPU layer optimizes GPU performance in serverless computing via GPU context caching and locality-aware scheduling to mitigate cold-starts and unnecessary data movement. The container subsystem accelerates the entire DL lifecycle by exploiting DL model structures and pipelined model loading to parallelize CPU-to-GPU memory copy and model execution. The subsystem exploits model partitioning techniques to accelerate the hybrid CPU/GPU architecture by dynamically distributing the DL model partitions to CPU and GPU. The scientific knowledge and tools designed and implemented from this research project will provide and enable innovations for next-generation cloud computing and deep learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项是根据2021年《美国救援计划法》的全部或部分资助（公共法117-2）.Next-Generation无服务器云计算为开发人员提供了简化对服务器管理和管理的访问，包括事件驱动的执行，良好的粒度资源提供，自动规模，自动规模和付费付费。机器学习社区正在利用无服务器云计算的这些好处，以简化深度学习（DL）应用程序的开发和部署。但是，现有的无服务器计算平台缺乏对GPU的有效支持，阻碍了DL从业人员将无服务器计算用于大规模应用程序。该项目将使用混合CPU/GPU体系结构开发一个有效的无服务器计算平台，以加速DL应用程序开发和部署。目的是提高深度学习和无服务器计算的尖端方法，这将导致巨大的飞跃，从而使DL从业人员，DL用户和云计算基础架构的提供者受益，从而为社会的科学发展做出贡献。研究结果还将在分布式计算，云计算和深度学习的交集中以令人兴奋的示例和现实世界系统的示例来增强本科和研究生教育。该项目将开发一个新型的无服务器计算平台，并具有混合CPU/GPU架构，该平台将提供与本机GPU相关的DL应用程序。两个核心组件构成了混合无服务器计算体系结构，垫片虚拟化的GPU（VGPU）层和一个重构的容器子系统。 SHIM VGPU层可实现高性能的GPU共享，该共享无服务的无服务器功能，具有低延迟和高可扩展性。该层通过使用API​​远程技术从无服务器函数拦截GPU调用来提供细粒度的GPU资源提供和性能隔离。 VGPU层通过GPU上下文缓存和局部感知的计划在无服务器计算中优化了GPU性能，以减轻冷启动和不必要的数据移动。容器子系统通过利用DL模型结构和管道模型加载来并行化CPU到GPU内存副本和模型执行来加速整个DL生命周期。子系统利用模型分区技术来通过将DL模型分区动态分配给CPU和GPU来加速混合CPU/GPU架构。该研究项目设计和实施的科学知识和工具将为下一代云计算和深度学习提供创新。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的智力优点和更广泛影响的评估评估标准的评估值得支持的。

项目成果

Accelerating Serverless Computing by Harvesting Idle Resources

• DOI: 10.1145/3485447.3511979
• 发表时间: 2021-08
• 期刊: Proceedings of the ACM Web Conference 2022
• 作者: Hanfei Yu;Hao Wang;Jian Li;Xuemei Yuan;Seung-Jong Park

Libra: Harvesting Idle Resources Safely and Timely in Serverless Clusters

• DOI: 10.1145/3588195.3592996
• 发表时间: 2023-08
• 期刊: Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Hanfei Yu;Christian Fontenot;Hao Wang;Jian Li;Xu Yuan;Seung-Jong Park