CAREER: Towards a High-Fidelity Knowledge Plane for Data-Center Networks

许多企业日益增长的计算和数据分析需求正越来越多地通过公司数据中心或最近基于“云”的数据中心的并行和分布式处理得到满足。 许多数据中心应用（例如，高性能计算应用）的性能直接取决于底层网络性能;因此，有效地管理数据中心网络（DCN）性能至关重要。 管理DCN性能是具有挑战性的，因为许多DCN应用（诸如基于云的Web服务、存储应用、高性能计算和金融交易应用）需要几十微秒量级的延迟，而ISP网络应用需要几百毫秒内的延迟保证。 因此，单独用于监测ISP网络的工具和技术不足以在DCN上下文中进行高保真度测量。智力优势：这个项目的目标是研究新的工具和技术，这将有助于建立一个高保真的知识平面的数据中心。 具体而言，知识平面将通过为DCN交换机配备低成本原语来收集高保真度测量，以直接测量延迟和丢失属性。 知识平面将允许运营商以准确和自动化的方式在现代数据中心网络中执行故障诊断、服务水平协议（SLA）监控、流量工程、网络配置和其他此类管理任务。 数据中心网络性能的最新知识还将有助于构建有效的调度和性能感知的作业放置算法，以提高各种DC应用程序的性能。 该项目将研究以下关键构建块的知识平面的建设：（1）它将开发新的可扩展的原语的延迟和损失的测量，可以在交换机中实现高速。它将侧重于跨交换机内的接口对以及跨交换机（它们之间可能有多条路径）的测量。(2)它将探索在多租户环境中新的按流或按类区分的延迟测量，这些环境需要基于每个客户的SLA保证。 它还将调查可扩展的原语，以获得每个数据包的测量交换机，这些原语将一起改造的调试和测量支持管理关键任务的DCN。 (3)为了确保知识平面的可扩展性，该项目还将研究测量信息的可扩展导出机制，包括用于查询网络设备信息的原语，而不是基于“推”的方法。更广泛的影响：该项目的研究成果将有助于大大简化数据中心应用程序的管理，从而降低成本，提高数据中心服务的可靠性和性能。 这个项目的结果是及时的，云计算准备在大规模多租户云中带来分布式和并行计算的复兴。 虽然这个建议本身是集中在DCN上下文中，开发的技术是相当普遍的，也将有助于调试ISP网络的性能。这项研究的结果将纳入本科和研究生课程工作。 作为该项目的一部分开发的所有教育材料和研究原型（软件和硬件）将公开共享，供其他研究人员使用。该项目还将吸引少数民族和妇女的参与。

The increased computational and data analysis demands of many enterprises are increasingly being met through parallel and distributed processing in corporate data centers (DCs), or more recently in `cloud'-based DCs. The performance of many DC applications (e.g, high-performance computing applications) directly depends on the underlying network performance; managing data center network (DCN) performance efficiently is therefore of utmost importance. Managing DCN performance is challenging because many DCN applications such as cloud-based Web services, storage applications, high performance computing, and financial trading applications require latencies of the order of 10s of microseconds, in contrast with ISP network applications that require latency guarantees within few 100s of milliseconds. Thus, tools and techniques developed for monitoring ISP networks alone are insufficient for high-fidelity measurements in the DCN context. Intellectual Merit: The goal of this project is to investigate novel tools and techniques that will help build a high-fidelity knowledge plane for data centers. Specifically, the knowledge plane will collect high-fidelity measurements by equipping DCN switches with low-cost primitives for direct measurement of latency and loss properties. The knowledge plane will allow operators to perform fault diagnosis, service level agreement (SLA) monitoring, traffic engineering, network provisioning and other such management tasks in modern data center networks in an accurate and automated fashion. The up-to-date knowledge of the data center network performance will also help in building effective scheduling and performance-aware job placement algorithms to improve the performance of various DC applications. The project will investigate the following key building blocks for the construction of the knowledge plane: (1) It will develop novel scalable primitives for latency and loss measurements that can be implemented in switches at high speed. It will focus on measurements across interfaces pairs within a switch as well as across switches that may have multiple paths between them. (2) It will explore novel per-flow or per-class differentiated latency measurements in multi-tenant environments that require SLA guarantees on a per-customer basis. It will also investigate scalable primitives for obtaining per-packet measurements in switches; together these primitives will be transformational in debugging and measurement support in managing mission-critical DCNs. (3) To ensure scalability of the knowledge plane; the project will also investigate mechanisms for scalable export of measurement information, including primitives for querying network devices for information as opposed to a `push'-based approach.Broader Impact: The research outcomes of this project will help ease the management of data center applications significantly, thereby reducing cost and increasing the reliability and performance of data center services. The results of this project are timely as well as cloud computing is poised to bring about a renaissance in distributed and parallel computing in large-scale multi-tenant clouds. While this proposal itself is focused on the DCN context, the techniques developed are quite general and will also help debugging performance in ISP networks. The results from this research will be incorporated into undergraduate and graduate course work. All education materials and research prototypes (both software and hardware) developed as part of this project will be shared publicly for other researchers to use. The project will also involve participation of minorities and women.