SHF: Small: Lightweight Virtualization Driven Elastic Memory Management and Cluster Scheduling

数据中心正在演变为在共享群集上托管异类工作负载，以降低运营成本并实现高资源利用率。然而，在异类硬件上调度具有不同资源要求和性能约束的异类工作负载是具有挑战性的。数据并行处理经常受到干扰和巨大的内存压力，导致过多的垃圾回收和内存不足错误，从而损害应用程序的性能和可靠性。集群内存管理和调度效率仍然较低，导致利用率低，多服务支持能力差。现有的方法要么关注应用感知，要么关注操作系统感知，因此不能很好地解决应用运行时和操作系统之间的语义鸿沟。该项目旨在通过轻量级的虚拟化弹性内存管理和集群调度来提升应用性能和集群效率。它将系统实验与严谨的设计和分析相结合，以提高性能和效率，并解决数据并行处理的存储压力。开发的系统软件将是开源的，提供机会培育一个跨越系统软件提供商和客户的大型生态系统。在轻量级容器的支持下，集群调度和底层操作系统可以协同工作，使得应用的动态资源需求可以暴露给操作系统，并且集群存储器管理器和调度器可以利用从性能计数器和操作系统检索到的丰富的运行时信息来辅助。为此，本项目的目标是为数据并行程序设计一种分布式内存管理器，该管理器能够在内存压力下生存并通过架构感知的容器放置来实现弹性集群内存管理，通过扩展Linux内核中现有的虚拟内存回收机制来设计协作分页来提高内存交换的性能，通过一种新的服务来检测和利用过度承诺机会来实现内存过量使用，并设计基于多队列的分布式任务调度器来管理性能干扰和硬件异构性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，认为值得支持。

Data-centers are evolving to host heterogeneous workloads on shared clusters to reduce the operational cost and achieve high resource utilization. However, it is challenging to schedule heterogeneous workloads with diverse resource requirements and performance constraints on heterogeneous hardware. Data parallel processing often suffers from interference and significant memory pressure, resulting in excessive garbage collection and out-of-memory errors that harm application performance and reliability. Cluster memory management and scheduling is still inefficient, leading to low utilization and poor multi-service support. Existing approaches either focus on application awareness or operating system awareness, thus are not well positioned to address the semantic gap between application run-times and the operating system. This project aims to improve application performance and cluster efficiency via lightweight virtualization-enabled elastic memory management and cluster scheduling. It combines system experimentation with rigorous design and analyses to improve performance and efficiency, and tackle memory pressure of data-parallel processing. Developed system software will be open-sourced, providing opportunities to foster a large ecosystem that spans system software providers and customers. Enabled by lightweight containers, cluster scheduling and the underlying operating system can cooperate synergistically, such that, the dynamic resource demand of an application can be exposed to the operating system, and the cluster memory manager and scheduler can be assisted with rich run-time information retrieved from performance counters and operating system. Towards this end, the project aims to devise a distributed memory manager for data-parallel programs that can survive from memory pressure and enable elastic cluster memory management with architecture-aware container placement, design a cooperative paging to improve performance of memory swapping by extending the current virtual memory reclaim mechanism in Linux kernel, enable memory over-commitment for elastic cluster scheduling with a new service that can detect and exploit the over-commitment opportunities, and design a multi-queue based distributed task scheduler to manage performance interference and hardware heterogeneity. The contributions include a library of developed mechanisms and open-source system software at cluster and kernel levels that can significantly improve cluster utilization and application performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.