CRII: OAC: High-Efficiency Serverless Computing Systems for Deep Learning: A Hybrid CPU/GPU Architecture

CRII：OAC：用于深度学习的高效无服务器计算系统：混合 CPU/GPU 架构

• 批准号: 2153502
• 负责人: Hao Wang
• 金额: $ 17.49万
• 依托单位: Louisiana State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-05-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2153502&HistoricalAwards=false
• 关键词: CRII; OAC; Efficiency; Serverless; Computing

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Next-generation serverless cloud computing provides developers with simplified access to server management and administration, including event-driven execution, fine-grained resource provisioning, auto-scaling, and pay-as-you-go billing. The machine learning community is taking advantage of these benefits of serverless cloud computing to ease the development and deployment of deep learning (DL) applications. However, existing serverless computing platforms lack efficient support for GPUs, impeding DL practitioners from utilizing serverless computing for large-scale applications. This project will develop an efficient serverless computing platform with a hybrid CPU/GPU architecture to accelerate DL application development and deployment. The goal is to advance cutting-edge methodologies in both deep learning and serverless computing, which will result in a significant leap forward to benefit DL practitioners, DL users, and providers of cloud computing infrastructures, contributing to science advancement for society. The research findings will also enhance undergraduate and graduate education with exciting examples and demonstrations of real-world systems at the intersection of distributed computing, cloud computing, and deep learning.The project will develop a novel serverless computing platform with a hybrid CPU/GPU architecture that will provide DL applications with native GPU performance. Two core components constitute the hybrid serverless computing architecture, a shim virtualized GPU (vGPU) layer and a refactored container subsystem. The shim vGPU layer enables high-performance GPU sharing for concurrent serverless functions with low latency and high scalability. This layer provides fine-grained GPU resource provisioning and performance isolation by intercepting GPU calls from serverless functions using API remoting techniques. The vGPU layer optimizes GPU performance in serverless computing via GPU context caching and locality-aware scheduling to mitigate cold-starts and unnecessary data movement. The container subsystem accelerates the entire DL lifecycle by exploiting DL model structures and pipelined model loading to parallelize CPU-to-GPU memory copy and model execution. The subsystem exploits model partitioning techniques to accelerate the hybrid CPU/GPU architecture by dynamically distributing the DL model partitions to CPU and GPU. The scientific knowledge and tools designed and implemented from this research project will provide and enable innovations for next-generation cloud computing and deep learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项的全部或部分资金来自《2021年美国救援计划法案》(Public Law 117-2)。下一代无服务器云计算为开发人员提供了对服务器管理和管理的简化访问，包括事件驱动执行、细粒度资源配置、自动扩展和即付即用计费。机器学习社区正在利用无服务器云计算的这些优势来简化深度学习(DL)应用程序的开发和部署。然而，现有的无服务器计算平台缺乏对GPU的有效支持，阻碍了数字图书馆从业者将无服务器计算用于大规模应用。该项目将开发一个高效的无服务器计算平台，采用CPU/GPU混合架构，以加快DL应用的开发和部署。目标是推动深度学习和无服务器计算领域的尖端方法，这将导致重大飞跃，使数字图书馆从业者、数字图书馆用户和云计算基础设施提供商受益，为社会的科学进步做出贡献。研究成果还将加强本科和研究生教育，提供令人兴奋的实例和现实世界系统的演示，这些系统位于分布式计算、云计算和深度学习的交汇点。该项目将开发一种具有混合CPU/GPU架构的新型无服务器计算平台，将为数字图书馆应用程序提供原生GPU性能。两个核心组件构成了混合无服务器计算体系结构，填隙虚拟GPU(VGPU)层和重构容器子系统。填补vGPU层支持高性能GPU共享，支持低延迟和高可伸缩性的并发无服务器功能。这一层使用API远程处理技术拦截来自无服务器函数的GPU调用，从而提供细粒度的GPU资源供应和性能隔离。VGPU层通过GPU环境缓存和位置感知调度来优化无服务器计算中的GPU性能，以减少冷启动和不必要的数据移动。容器子系统通过利用DL模型结构和流水线模型加载来并行CPU到GPU的内存复制和模型执行，从而加速了整个DL生命周期。该子系统利用模型分区技术，通过将DL模型分区动态分配给CPU和GPU来加速混合CPU/GPU体系结构。该研究项目设计和实施的科学知识和工具将为下一代云计算和深度学习提供创新并使之成为可能。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Accelerating Serverless Computing by Harvesting Idle Resources

• DOI: 10.1145/3485447.3511979
• 发表时间: 2021-08
• 期刊: Proceedings of the ACM Web Conference 2022
• 作者: Hanfei Yu;Hao Wang;Jian Li;Xuemei Yuan;Seung-Jong Park

Libra: Harvesting Idle Resources Safely and Timely in Serverless Clusters

• DOI: 10.1145/3588195.3592996
• 发表时间: 2023-08
• 期刊: Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Hanfei Yu;Christian Fontenot;Hao Wang;Jian Li;Xu Yuan;Seung-Jong Park