SHF: Small: Lightweight Virtualization Driven Elastic Memory Management and Cluster Scheduling

数据中心正在向在共享集群上托管异构工作负载的方向发展，以降低运营成本并实现高资源利用率。然而，在异构硬件上调度具有不同资源需求和性能约束的异构工作负载是一项挑战。数据并行处理经常受到干扰和巨大的内存压力，导致过多的垃圾收集和内存不足错误，从而损害应用程序的性能和可靠性。集群内存管理和调度仍然效率低下，导致低利用率和较差的多服务支持。现有的方法要么关注应用程序感知，要么关注操作系统感知，因此不能很好地解决应用程序运行时和操作系统之间的语义差距。该项目旨在通过支持轻量级虚拟化的弹性内存管理和集群调度来提高应用程序性能和集群效率。它将系统实验与严格的设计和分析相结合，以提高性能和效率，并解决数据并行处理的内存压力。开发的系统软件将是开源的，为培育一个跨越系统软件供应商和客户的大型生态系统提供了机会。在轻量级容器的支持下，集群调度和底层操作系统可以协同合作，这样，应用程序的动态资源需求就可以向操作系统公开，集群内存管理器和调度器可以通过从性能计数器和操作系统检索到的丰富运行时信息得到帮助。为此，该项目旨在为数据并行程序设计一个分布式内存管理器，该管理器可以在内存压力下生存，并支持具有架构感知容器放置的弹性集群内存管理，设计一个协作分页，通过扩展Linux内核中当前的虚拟内存回收机制来提高内存交换性能。为弹性集群调度启用内存超分配，使用可以检测和利用超分配机会的新服务，并设计基于多队列的分布式任务调度器来管理性能干扰和硬件异构性。这些贡献包括在集群和内核级别开发的机制和开源系统软件库，可以显著提高集群利用率和应用程序性能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data-centers are evolving to host heterogeneous workloads on shared clusters to reduce the operational cost and achieve high resource utilization. However, it is challenging to schedule heterogeneous workloads with diverse resource requirements and performance constraints on heterogeneous hardware. Data parallel processing often suffers from interference and significant memory pressure, resulting in excessive garbage collection and out-of-memory errors that harm application performance and reliability. Cluster memory management and scheduling is still inefficient, leading to low utilization and poor multi-service support. Existing approaches either focus on application awareness or operating system awareness, thus are not well positioned to address the semantic gap between application run-times and the operating system. This project aims to improve application performance and cluster efficiency via lightweight virtualization-enabled elastic memory management and cluster scheduling. It combines system experimentation with rigorous design and analyses to improve performance and efficiency, and tackle memory pressure of data-parallel processing. Developed system software will be open-sourced, providing opportunities to foster a large ecosystem that spans system software providers and customers. Enabled by lightweight containers, cluster scheduling and the underlying operating system can cooperate synergistically, such that, the dynamic resource demand of an application can be exposed to the operating system, and the cluster memory manager and scheduler can be assisted with rich run-time information retrieved from performance counters and operating system. Towards this end, the project aims to devise a distributed memory manager for data-parallel programs that can survive from memory pressure and enable elastic cluster memory management with architecture-aware container placement, design a cooperative paging to improve performance of memory swapping by extending the current virtual memory reclaim mechanism in Linux kernel, enable memory over-commitment for elastic cluster scheduling with a new service that can detect and exploit the over-commitment opportunities, and design a multi-queue based distributed task scheduler to manage performance interference and hardware heterogeneity. The contributions include a library of developed mechanisms and open-source system software at cluster and kernel levels that can significantly improve cluster utilization and application performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.