CNS Core: Small: A Multi-Stakeholder Integrated Approach to Reduce Tail Latency Using Heterogeneity

CNS 核心：小型：利用异构性减少尾部延迟的多利益相关者集成方法

• 批准号: 1909004
• 负责人: Timothy Zhu
• 金额: $ 50万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909004&HistoricalAwards=false
• 关键词: CNS; Core; Small; Multi; Stakeholder

There are many different types of computer hardware. Examples include mobile cell phone processors, machine learning accelerators, traditional server processors, gaming platforms, and so forth, but these resources are specialized and primarily utilized within a single domain. As cloud computing providers broaden their markets and start offering more types of hardware, it opens an opportunity to design heterogeneity-aware computing systems, which combine different types of resources that are synergistic. This research will investigate techniques for intentionally utilizing heterogeneity as a controllable parameter for improving performance. Specifically, the research will focus on the tail latency performance metric, which has been identified by industry as an important performance metric affecting the responsiveness of user-facing Internet services. To deploy and harness the right heterogeneity at the right time, this research will consider the unique problems faced by three key stakeholders (Application Deployer, Resource Manager, and Infrastructure Provider). From the Application Deployer's perspective, the research will consider how to decompose an application into phases/sub-components that can benefit from different types of resources. From the Resource Manager's perspective, the research will determine the right quantity and mixture of resource types as well as how to schedule across these resources to minimize tail latency. From the Infrastructure Provider's perspective, the research will consider issues arising from sharing a mixture of different resources between multiple applications. Additionally, the research will leverage cross-stakeholder information towards a unified strategy for deploying and harnessing heterogeneity.This research will be applicable to both providers of data centers, such as cloud providers, as well as businesses that use that infrastructure. For the provider, the research can improve the performance and lower the cost of data centers, which are critical components of the national infrastructure and economy. For the user, the research can enable emerging interactive applications, such as complex data analytics and real-time machine learning, to achieve high performance at low cost. This research promotes using diverse mixtures of resources, which could spur the development of new types of hardware and software. In addition to the broader research impacts, there is a plan to enhance the undergraduate and graduate courses with ideas and software generated by this research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

有许多不同类型的计算机硬件。例子包括移动手机处理器、机器学习加速器、传统服务器处理器、游戏平台等等，但这些资源都是专门的，主要在单个领域内使用。随着云计算提供商扩大其市场并开始提供更多类型的硬件，它为设计异构感知计算系统提供了机会，这些系统可以结合不同类型的协同资源。本研究将探讨有意利用异质性作为提高性能的可控参数的技术。具体而言，研究将集中在尾部延迟性能指标上，该指标已被业界确定为影响面向用户的互联网服务响应性的重要性能指标。为了在正确的时间部署和利用正确的异构性，本研究将考虑三个关键涉众（应用程序部署者、资源管理器和基础设施提供者）所面临的独特问题。从应用程序部署者的角度来看，该研究将考虑如何将应用程序分解为可以从不同类型的资源中受益的阶段/子组件。从资源管理器的角度来看，研究将确定资源类型的正确数量和混合，以及如何跨这些资源调度以最小化尾部延迟。从基础设施提供商的角度来看，该研究将考虑在多个应用程序之间共享不同资源的混合所产生的问题。此外，该研究将利用跨利益相关者的信息来制定统一的战略，以部署和利用异质性。这项研究既适用于数据中心的提供商，如云提供商，也适用于使用该基础设施的企业。数据中心是国家基础设施和经济的重要组成部分，对于提供商来说，研究可以提高数据中心的性能并降低成本。对于用户来说，这项研究可以使新兴的交互式应用，如复杂的数据分析和实时机器学习，以低成本实现高性能。这项研究提倡使用不同的资源组合，这可能会刺激新型硬件和软件的发展。除了更广泛的研究影响之外，还有一个计划是用这项研究产生的想法和软件来增强本科和研究生课程。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Overflowing emerging neural network inference tasks from the GPU to the CPU on heterogeneous servers

• DOI: 10.1145/3534056.3534935
• 发表时间: 2022-06
• 期刊: Proceedings of the 15th ACM International Conference on Systems and Storage
• 作者: Adithya Kumar;A. Sivasubramaniam;T. Zhu

Kerveros: Efficient and Scalable Cloud Admission Control

Kerveros：高效且可扩展的云准入控制

• 发表时间: 2023
• 期刊: 17th USENIX Symposium on Operating Systems Design and Implementation
• 作者: Sajal, Sultan Mahmud;Marshall, Luke;Li, Beibin;Zhou, Shandan;Pan, Abhisek;Mellou, Konstantina;Narayanan, Deepak;Zhu, Timothy;Dion, David;Moscibroda, Thomas
• 通讯作者: Moscibroda, Thomas

SplitRPC: A {Control + Data} Path Splitting RPC Stack for ML Inference Serving

• DOI: 10.1145/3589974
• 发表时间: 2023-05
• 期刊: Proceedings of the ACM on Measurement and Analysis of Computing Systems
• 作者: Adithya Kumar;A. Sivasubramaniam;T. Zhu