CAREER: A Programmable Measurement Architecture for Network Operations

许多公司(如谷歌、微软、Facebook、亚马逊)投入巨资建设具有更高链接速度的更大数据中心网络。仔细管理这样的大型数据中心网络以支持多个租户、满足应用的服务级别协议并降低运营成本变得越来越重要。大多数网络管理任务，如流量计算、流量工程、负载平衡和性能诊断，都依赖于对整个网络中随时间变化的流量进行准确和及时的测量。遗憾的是，目前的测量支持存在三个关键问题：(1)网络设备厂商往往将测量视为二等公民，将大部分资源投入到满足网络控制功能(转发、防火墙、负载均衡等)上，留下有限的资源来支持测量。(2)操作员对测量什么(不测量)以及何时测量的控制有限。因此，本已有限的测量资源有时被浪费在测量运营商不关心的流量上，留下更少的资源用于测量重要的流量。(3)这些解决方案是面向设备的，而不是网络范围的。运营商不得不在多个设备上深入有限的测量支持，努力在线下了解这些测量结果，并发现回答他们的全网查询具有挑战性。该项目旨在设计和构建可编程测量体系结构图，以改变当今企业和数据中心网络中的网络测量实践。受软件定义的网络的启发，MAP允许运营商在控制器中灵活地对其全网测量查询进行编程。为了回答这些问题，MAP会在整个网络堆栈的不同位置自动配置和协调新的测量基元。MAP还允许运营商动态更改其查询并自动重新配置原语以处理网络动态。该项目有三个关键研究方向：首先，该项目将在整个网络堆栈(VM、Hypervisor、Switch和Packet Sniffer)中引入新的测量算法和设计。研究人员将重新设计这些设备上的测量原语，使其既能支持不同的测量要求，又能在有限的资源和能力下高效地提高数据包处理性能。其次，研究人员将为运营商设计新的声明性测量抽象，以明确表达网络层面的测量内容及其准确性/及时性要求。研究人员还将设计和实现一个运行时系统，该系统自动将测量抽象与设备上的原语相匹配，跨任务动态分配资源，并处理网络动态，如移动主机和路由更改。最后，研究人员将研究如何使用MAP？S原语、抽象和运行时来支持几个重要的、新颖的测量任务，并在MAP上对它们进行评估。这项拟议的研究如果成功，将从根本上改变企业和数据中心的网络测量和管理实践，并导致新的网络设备设计。这项研究将促进理论、编程语言和网络之间的跨学科研究。该项目还有一个重要的教育部分，包括设计和创新研究生课程和关于软件定义的网络和计算机网络的课程材料，以及为本科生和代表性不足群体的成员提供研究经验。

Many companies (e.g., Google, Microsoft, Facebook, Amazon) put huge capital investments into building larger data center networks with higher link speeds. It becomes increasingly important to carefully manage such large data center networks to support multiple tenants, meet service level agreements of applications, and reduce operation cost. Most network management tasks, such as traffic accounting, traffic engineering, load balancing, and performance diagnosis, rely on accurate and timely measurement of time-varying traffic across the entire network. Unfortunately, there are three key problems in today's measurement support: (1) Network device vendors often treat measurement as a second-class citizen, devoting most of the resources to meet the networking control functions (forwarding, firewalls, load balancing, etc.), leaving limited resources for supporting measurement. (2) Operators have limited control over what (not) to measure and when to measure. As a result, the already limited measurement resources are sometimes wasted to measure flows that operators do not care, leaving even fewer resources for measuring important flows. (3) These solutions are device oriented instead of network wide. Operators have to dive into the limited measurement support at multiple devices, taking great efforts to understand these measurement results offline, and find it challenging to answer their network-wide queries.This project aims to design and build a programmable measurement architecture MAP, which transforms today's network measurement practice in enterprise and data center networks. Inspired by software-defined networking, MAP allows operators to flexibly program their network-wide measurement queries in a controller. To answer these queries, MAP automatically configures and coordinates new measurement primitives at different places across the network stack. MAP also allows operators to dynamically change their queries and automatically reconfigure the primitives accordingly to handle network dynamics.There are three key research directions: First, the project will bring novel measurement algorithms and designs throughout the network stack (VMs, hypervisors, switches, and packet sniffers). The researcher will redesign the measurement primitives at these devices to make them both generic in supporting diverse measurement requirements, and efficient in packet processing performance with limited resources and capabilities. Second, the researcher will design new declarative measurement abstractions for operators to clearly express what to measure at the network level and their accuracy/timeliness requirements. The researcher will also design and implement a runtime system that automatically matches the measurement abstractions with primitives at devices, dynamically allocates resources across tasks and handles network dynamics such as mobile hosts and routing changes. Finally, the researcher will study how to support several important and novel measurement tasks with MAP?s primitives, abstractions, and runtime, and evaluate them on MAP. The proposed research, if successful, will fundamentally change network measurement and management practice in enterprise and data centers, and lead to new designs of network devices. The research will facilitate inter-discipline research between theory, programming languages, and networking. The project also has a significant education component, including the design and innovations of graduate courses and course material on software-defined networking and computer networking, as well as research experiences for undergraduate students and members of under-represented groups.