SHF: Small: Lightweight Virtualization Driven Elastic Memory Management and Cluster Scheduling

SHF：小型：轻量级虚拟化驱动的弹性内存管理和集群调度

• 批准号: 1816850
• 负责人: Xiaobo Zhou
• 金额: $ 37.6万
• 依托单位: University of Colorado at Colorado Springs
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816850&HistoricalAwards=false
• 关键词: SHF; Small; Lightweight; Virtualization; Driven

Data-centers are evolving to host heterogeneous workloads on shared clusters to reduce the operational cost and achieve high resource utilization. However, it is challenging to schedule heterogeneous workloads with diverse resource requirements and performance constraints on heterogeneous hardware. Data parallel processing often suffers from interference and significant memory pressure, resulting in excessive garbage collection and out-of-memory errors that harm application performance and reliability. Cluster memory management and scheduling is still inefficient, leading to low utilization and poor multi-service support. Existing approaches either focus on application awareness or operating system awareness, thus are not well positioned to address the semantic gap between application run-times and the operating system. This project aims to improve application performance and cluster efficiency via lightweight virtualization-enabled elastic memory management and cluster scheduling. It combines system experimentation with rigorous design and analyses to improve performance and efficiency, and tackle memory pressure of data-parallel processing. Developed system software will be open-sourced, providing opportunities to foster a large ecosystem that spans system software providers and customers. Enabled by lightweight containers, cluster scheduling and the underlying operating system can cooperate synergistically, such that, the dynamic resource demand of an application can be exposed to the operating system, and the cluster memory manager and scheduler can be assisted with rich run-time information retrieved from performance counters and operating system. Towards this end, the project aims to devise a distributed memory manager for data-parallel programs that can survive from memory pressure and enable elastic cluster memory management with architecture-aware container placement, design a cooperative paging to improve performance of memory swapping by extending the current virtual memory reclaim mechanism in Linux kernel, enable memory over-commitment for elastic cluster scheduling with a new service that can detect and exploit the over-commitment opportunities, and design a multi-queue based distributed task scheduler to manage performance interference and hardware heterogeneity. The contributions include a library of developed mechanisms and open-source system software at cluster and kernel levels that can significantly improve cluster utilization and application performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据中心正在发展到在共享集群上托管异构工作负载，以降低运营成本并实现高资源利用率。然而，它是具有挑战性的调度异构工作负载与不同的资源需求和性能约束异构硬件。数据并行处理经常受到干扰和巨大的内存压力，导致过多的垃圾收集和内存不足错误，从而损害应用程序的性能和可靠性。集群内存管理和调度仍然效率低下，导致利用率低和多业务支持差。现有的方法要么专注于应用程序感知，要么专注于操作系统感知，因此不能很好地解决应用程序运行时和操作系统之间的语义差距。该项目旨在通过轻量级虚拟化支持的弹性内存管理和集群调度来提高应用程序性能和集群效率。它将系统实验与严格的设计和分析相结合，以提高性能和效率，并解决数据并行处理的内存压力。开发的系统软件将是开源的，为培育一个涵盖系统软件提供商和客户的大型生态系统提供了机会。通过轻量级容器启用，集群调度和底层操作系统可以协同合作，使得应用的动态资源需求可以暴露给操作系统，并且集群存储器管理器和调度器可以利用从性能计数器和操作系统检索的丰富运行时信息来辅助。 为此，本项目的目标是设计一个分布式内存管理器，用于数据并行程序，使其能够承受内存压力，并通过体系结构感知的容器放置实现弹性的集群内存管理;设计一个协作分页，通过扩展Linux内核中现有的虚拟内存回收机制来提高内存交换的性能;通过一种新的服务来检测和利用内存过度使用的机会，实现弹性集群调度的内存过度使用，并设计了一种基于多队列的分布式任务调度器来管理性能干扰和硬件异构性。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Elastic Parameter Server: Accelerating ML Training With Scalable Resource Scheduling

• DOI: 10.1109/tpds.2021.3104242
• 发表时间: 2022-05
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Shaoqi Wang;Aidi Pi;Xiaobo Zhou

Semantic-aware Workflow Construction and Analysis for Distributed Data Analytics Systems

• DOI: 10.1145/3307681.3325404
• 发表时间: 2019-06
• 期刊: Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Aidi Pi;Wei Chen-;Shaoqi Wang;Xiaobo Zhou

Memory at your service: fast memory allocation for latency-critical services

• DOI: 10.1145/3464298.3493394
• 发表时间: 2021-09
• 期刊: Proceedings of the 22nd International Middleware Conference
• 作者: Aidi Pi;Junxian Zhao;Shaoqi Wang;Xiaobo Zhou

OS-Augmented Oversubscription of Opportunistic Memory with a User-Assisted OOM Killer

• DOI: 10.1145/3361525.3361534
• 发表时间: 2019-12
• 期刊: Proceedings of the 20th International Middleware Conference
• 作者: Wei Chen;Aidi Pi;Shaoqi Wang;Xiaobo Zhou

FlashByte: Improving Memory Efficiency with Lightweight Native Storage

FlashByte：通过轻量级本机存储提高内存效率

• DOI: 10.1109/ccgrid51090.2021.00016
• 发表时间: 2021
• 期刊: Cloud and Internet Computing (CCGrid
• 作者: Zhao, Junxian;Pi, Aidi;Wang, Shaoqi;Zhou, Xiaobo
• 通讯作者: Zhou, Xiaobo