CAREER: QoS-aware Systems for Accelerated Datacenters

该项目解决了提供足够和高效的基础设施以满足人工智能(AI)和机器学习(ML)应用程序日益增长的计算需求的挑战。为了支持日益流行的ML和AI应用程序，数据中心正在整合专门的加速器硬件进行计算，因为最近的研究表明，传统的基于CPU的基础设施的效率比这些基于加速器的设计低100倍。然而，尽管社区在很大程度上关注硬件设计，但目前很少有研究关注设计系统软件来管理硬件加速数据中心的性能和效率。目前的数据中心系统软件大多是针对CPU平台量身定制的。CPU和加速器之间存在着重大而根本的差异，这些差异会影响系统软件堆栈中设计决策的有效性。这项工作重新思考和重新设计了系统软件堆栈，以适应数据中心硬件加速的新兴格局。这项工作的目标是重新设计数据中心系统，以支持大规模加速，以满足未来的计算需求。该项目设计了系统软件，在由通用处理器和各种加速器组成的异构平台上调度和分配资源，管理服务质量(Qos)，实现高效率。为此，该项目重点关注创新的三大支柱。首先，本项目设计了针对异构型加速基础设施的簇级调度器，以准确预测性能干扰，在不违反服务质量的前提下最大化利用率。其次，该项目设计了一个应用级加速管理器，该管理器可以准确识别瓶颈服务，使用不同的加速策略评估潜在的改进，并在服务阶段动态(重新)分配加速器资源，从而显著改善延迟。第三，该项目为加速平台设计了一个节点级资源管理器，该管理器管理加速器上独特的共享和重新配置行为，以确保服务质量和高吞吐量。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project addresses the challenge of providing sufficient and efficient infrastructure to meet the increasing computational demands of Artificial Intelligence (AI) and Machine Learning (ML) applications. To support the ever-growing prevalence of ML and AI applications, data centers are incorporating specialized accelerator hardware for computation as recent work has shown that traditional CPU-based infrastructures are up to 100x less efficient than these accelerator-based designs. However, while the community has largely focused on hardware designs, currently there is very little research attention on designing system software to manage the performance and efficiency for hardware accelerated data centers. Current data center system software is mostly tailored to CPU platforms. There are significant and fundamental differences between CPUs and accelerators that impact the effectiveness of design decisions in the system software stack. This work rethinks and redesigns the system software stack for the emerging landscape of hardware acceleration in data centers. The goal of this work is to redesign datacenter systems to support acceleration at scale to meet the future computational demand. The project designs system software to schedule and allocate resources among heterogeneous platforms composed of both general purpose processors and various accelerators, managing quality of service (QoS) and achieving high efficiency. To this end, the project focuses on three pillars of innovation. First, the project designs the cluster-level scheduler for heterogeneous accelerated infrastructures to precisely predict the performance interference and maximizes the utilization without violating QoS based on such prediction. Second, the project designs an application-level acceleration manager that accurately identifies the bottleneck service, estimates potential improvement using different acceleration strategies and dynamically (re)allocates the accelerator resources across service stages, resulting in significantly improved latency. Third, the project designs a node-level resource manager for accelerated platforms that manages unique sharing and reconfiguration behaviors on accelerators to ensure QoS and high throughput.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.