CNS Core: Small: A Multi-Stakeholder Integrated Approach to Reduce Tail Latency Using Heterogeneity

有许多不同类型的计算机硬件。例如移动电话处理器、机器学习加速器、传统服务器处理器、游戏平台等，但这些资源是专门的，主要在单个域中使用。随着云计算提供商拓宽市场，开始提供更多类型的硬件，这为设计具有异构性的计算系统打开了机会，这些系统结合了协同的不同类型的资源。这项研究将研究有意利用异构性作为提高性能的可控参数的技术。具体地说，研究将重点放在尾部延迟性能指标上，该指标已被业界确定为影响面向用户的互联网服务响应速度的重要性能指标。为了在正确的时间部署和利用正确的异构性，本研究将考虑三个关键利益相关者(应用程序部署人员、资源管理人员和基础设施提供商)面临的独特问题。从应用程序部署人员的角度来看，研究将考虑如何将应用程序分解为可以从不同类型的资源中受益的阶段/子组件。从资源管理器的角度来看，研究将确定正确的资源类型数量和混合，以及如何跨这些资源进行调度以最大限度地减少尾部延迟。从基础设施提供商的角度来看，研究将考虑在多个应用程序之间共享不同资源的混合所产生的问题。此外，该研究将利用跨利益相关者信息制定部署和利用异构性的统一战略。该研究将适用于数据中心提供商(如云提供商)以及使用该基础设施的企业。对于提供商来说，研究可以提高数据中心的性能并降低成本，数据中心是国家基础设施和经济的关键组成部分。对于用户来说，研究可以使新兴的交互应用程序，如复杂数据分析和实时机器学习，以低成本实现高性能。这项研究促进了资源的多样化组合，这可能会刺激新型硬件和软件的发展。除了更广泛的研究影响，还有一项计划，用这项研究产生的想法和软件来增强本科生和研究生的课程。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

There are many different types of computer hardware. Examples include mobile cell phone processors, machine learning accelerators, traditional server processors, gaming platforms, and so forth, but these resources are specialized and primarily utilized within a single domain. As cloud computing providers broaden their markets and start offering more types of hardware, it opens an opportunity to design heterogeneity-aware computing systems, which combine different types of resources that are synergistic. This research will investigate techniques for intentionally utilizing heterogeneity as a controllable parameter for improving performance. Specifically, the research will focus on the tail latency performance metric, which has been identified by industry as an important performance metric affecting the responsiveness of user-facing Internet services. To deploy and harness the right heterogeneity at the right time, this research will consider the unique problems faced by three key stakeholders (Application Deployer, Resource Manager, and Infrastructure Provider). From the Application Deployer's perspective, the research will consider how to decompose an application into phases/sub-components that can benefit from different types of resources. From the Resource Manager's perspective, the research will determine the right quantity and mixture of resource types as well as how to schedule across these resources to minimize tail latency. From the Infrastructure Provider's perspective, the research will consider issues arising from sharing a mixture of different resources between multiple applications. Additionally, the research will leverage cross-stakeholder information towards a unified strategy for deploying and harnessing heterogeneity.This research will be applicable to both providers of data centers, such as cloud providers, as well as businesses that use that infrastructure. For the provider, the research can improve the performance and lower the cost of data centers, which are critical components of the national infrastructure and economy. For the user, the research can enable emerging interactive applications, such as complex data analytics and real-time machine learning, to achieve high performance at low cost. This research promotes using diverse mixtures of resources, which could spur the development of new types of hardware and software. In addition to the broader research impacts, there is a plan to enhance the undergraduate and graduate courses with ideas and software generated by this research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.