CAREER: Architecting Datacenters for Optimized Tail Latency at Scale

现代社会严重依赖云计算和各种各样的在线服务，如社交媒体、网络搜索、数字内容交付等。这些实用程序数十亿美元的市场是通过大规模部署服务器（即企业级计算机）（即数据中心）来实现的。这类部署在数据中心的服务必须可靠地为每个用户请求生成高质量、低延迟的响应，因为即使是不常见的性能或可用性中断也会导致重大的收入损失。因此，服务提供者设置严格的服务水平目标（service Level Objectives, slo），定义每个服务组件的响应延迟分布的尾端的可接受行为。遵守这样的slo是具有挑战性的，因为计算系统通常是为了满足平均性能目标而不是尾部性能目标而优化的。第二个关键挑战来自于需要跟上下一代服务日益增长的需求。服务器不断地通过数据中心的内部网络进行通信，以便在功能、用户和数据集方面协同支持所需规模的在线服务。随着这三个方面的需求不断增长，数据中心内的服务器间数据移动据报道每12-15个月翻一番，因此成为主要的性能决定因素。该项目旨在为未来数据中心的设计提供信息，使其能够跟上推动现代数字经济的下一代在线服务不断增长的需求。实现这一目标的两种主要方法是：(i)通过整体跨组件（计算、内存和网络）设计优化大幅提高数据中心内部数据移动效率，以及（ii）开发嵌入在每个关键系统组件中的慢速感知机制，以满足数据中心环境特有的性能指标。该项目追求两个主要途径，以提高下一代数据中心的效率，发挥其提供高质量在线服务的关键作用。第一个途径研究了从回顾性评估指标到从集群规模到微架构的普遍的、整体的优化旋旋式提升慢速意识的潜力。将开发一个整体的慢速感知优化框架，以实现集群范围内的动态请求优先级策略。反过来，这些策略将驱动在性能关键型系统组件（包括计算、网络和内存资源）的底层硬件中实现的一组慢速感知机制。由于数据移动主要决定了计算效率，第二个研究途径提出了在集群规模和每个单独端点（即服务器）上减少数据移动的技术。随着网络功能的增长，服务器上的数据移动必须通过对每个服务器的网络接口和内存层次结构进行明智的协同设计来智能地进行编排。这两种研究途径结合起来，引入了一种新的大规模系统设计视角，其中整体系统设计方法实现了新的组件间协同作用，有望大幅提高端到端性能和效率。最后，计划研究的第三个目标，与两个主要研究途径相结合，是开发新的模拟方法和工具，以便在典型的学术环境中使用有限的计算资源来评估数据中心规模的技术，同时在模拟系统规模、速度和准确性之间取得平衡。协同数据中心主题的教育活动也将在研究生、本科生和K-12年级进行。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Modern societies heavily rely on cloud computing and a wide variety of online services, such as social media, web search, digital content delivery, etc. The multi-billion-dollar market of these utilities is enabled by massive deployments of servers (i.e., enterprise-grade computers), known as datacenters. It is imperative for such datacenter-deployed services to reliably generate high-quality, low-latency responses to every user request, as even infrequent performance or availability hiccups incur significant revenue loss. Therefore, service providers set strict Service Level Objectives (SLOs) that define the acceptable behavior of the tail-end of each service component's response latency distribution. Abiding by such SLOs is challenging, as computing systems have been conventionally optimized to meet average rather than tail performance goals. A second key challenge stems from the need to keep up with growing demands of next-generation services. Servers constantly communicate over the datacenter's internal network to collaboratively enable an online service at the required scale, in terms of features, users, and datasets. With demand relentlessly growing in each of these three dimensions, inter-server data movement within the datacenter is reportedly doubling every 12-15 months, thus becoming a major performance determinant. This project aims to inform the design of future datacenters that will be able to keep up with the growing demands of next-generation online services powering modern digital economies. The two primary approaches to achieve that are (i) drastic improvement of intra-datacenter data movement efficiency via holistic cross-component (compute, memory, and network) design optimizations, and (ii) development of SLO-aware mechanisms embedded within each key system component to natively cater to the performance metrics of interest that are unique to datacenter environments.This project pursues two main avenues to advance the efficiency of next-generation datacenters in their crucial role of delivering high-quality online services. The first avenue investigates the potential of promoting SLO-awareness from a retrospective evaluation metric to a pervasive, integral optimization knob from cluster scale down to microarchitecture. A holistic SLO-aware optimization framework will be developed to enable enforcement of cluster-wide dynamic request prioritization policies. In turn, these policies will drive a set of SLO-aware mechanisms implemented in the underlying hardware of performance-critical system components, including compute, network, and memory resources. Because data movement predominantly dictates computation efficiency, the second research avenue proposes techniques for reduced data movement both at the cluster scale and at each individual endpoint (i.e., server). As networking capabilities grow, on-server data movement must be intelligently orchestrated via judicious co-design of each server's network interface and memory hierarchy. The two research avenues combined introduce a new large-scale system design perspective, where a holistic system design approach enables new inter-component synergies that promise drastic improvements to end-to-end performance and efficiency. Finally, a third objective of the planned research, integral to the two main research avenues, is the development of new simulation methodologies and tools that enable evaluation of datacenter-scale techniques using limited compute resources attainable in typical academic environments, while striking a balance between simulated system scale, speed, and accuracy. Synergistic datacenter-themed educational activities will also be undertaken at the graduate, undergraduate, and K-12 levels.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.