CAREER: QoS-aware Systems for Accelerated Datacenters

该项目解决了提供足够和高效的基础设施以满足人工智能（AI）和机器学习（ML）应用程序日益增长的计算需求的挑战。为了支持不断增长的ML和AI应用，数据中心正在整合专门的加速器硬件进行计算，因为最近的工作表明，传统的基于cpu的基础设施的效率比这些基于加速器的设计低100倍。然而，当社区主要关注硬件设计时，目前很少有研究关注设计系统软件来管理硬件加速数据中心的性能和效率。目前的数据中心系统软件大多是针对CPU平台量身定制的。cpu和加速器之间存在显著的根本差异，这些差异会影响系统软件堆栈中设计决策的有效性。这项工作重新思考和重新设计了系统软件堆栈，以适应数据中心硬件加速的新兴景观。这项工作的目标是重新设计数据中心系统，以支持大规模加速，以满足未来的计算需求。本课题设计系统软件，在由通用处理器和各种加速器组成的异构平台之间调度和分配资源，管理服务质量（QoS），实现高效率。为此，该项目侧重于创新的三大支柱。首先，本项目针对异构加速基础设施设计集群级调度器，精确预测性能干扰，在不违背QoS的前提下实现利用率最大化。其次，该项目设计了一个应用级加速管理器，它可以准确识别瓶颈服务，使用不同的加速策略估计潜在的改进，并在服务阶段之间动态（重新）分配加速器资源，从而显著改善延迟。第三，本项目为加速平台设计了一个节点级资源管理器，用于管理加速器上独特的共享和重构行为，以确保QoS和高吞吐量。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project addresses the challenge of providing sufficient and efficient infrastructure to meet the increasing computational demands of Artificial Intelligence (AI) and Machine Learning (ML) applications. To support the ever-growing prevalence of ML and AI applications, data centers are incorporating specialized accelerator hardware for computation as recent work has shown that traditional CPU-based infrastructures are up to 100x less efficient than these accelerator-based designs. However, while the community has largely focused on hardware designs, currently there is very little research attention on designing system software to manage the performance and efficiency for hardware accelerated data centers. Current data center system software is mostly tailored to CPU platforms. There are significant and fundamental differences between CPUs and accelerators that impact the effectiveness of design decisions in the system software stack. This work rethinks and redesigns the system software stack for the emerging landscape of hardware acceleration in data centers. The goal of this work is to redesign datacenter systems to support acceleration at scale to meet the future computational demand. The project designs system software to schedule and allocate resources among heterogeneous platforms composed of both general purpose processors and various accelerators, managing quality of service (QoS) and achieving high efficiency. To this end, the project focuses on three pillars of innovation. First, the project designs the cluster-level scheduler for heterogeneous accelerated infrastructures to precisely predict the performance interference and maximizes the utilization without violating QoS based on such prediction. Second, the project designs an application-level acceleration manager that accurately identifies the bottleneck service, estimates potential improvement using different acceleration strategies and dynamically (re)allocates the accelerator resources across service stages, resulting in significantly improved latency. Third, the project designs a node-level resource manager for accelerated platforms that manages unique sharing and reconfiguration behaviors on accelerators to ensure QoS and high throughput.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.