IMR: MT: Tools for Programming Distributed Data-plane Measurements

了解关键网络上的流量-它由什么组成以及如何变化-对于改进现代信息服务至关重要。然而，传统上，研究人员很难开发新的工具来剖析这种流量并分析其特征，同时注意保护用户隐私。然而，最近，相对便宜的可编程交换机的发展使开发诊断工具成为可能，并将它们直接放置在网络中，在流量流动的路径上。在这种情况下，新工具有可能在所有互联网流量流经时看到它，例如，从大学校园到更广泛的互联网，或者沿着公司广域网或数据中心。不幸的是，尽管开发这类工具是可能的，但这样做目前是一个非常困难且容易出错的过程。为了改善这种情况，研究小组将开发一种新的编程语言和系统Lucid，它将促进在实时可编程网络中开发、调试和部署网络测量工具的过程。研究团队将提供一种编译器，将高级Lucid程序转换为在多个地方执行的低级代码-直接在可编程交换机上执行，或在连接到相关网络的服务器上作为支持执行。此外，该团队将提供一组可重复使用的组件，网络测量研究人员可以将这些组件插入到一起，以快速开始一个新的想法。为了帮助教授研究人员如何使用这种新语言，该团队正在为网络方面的主要会议开发教程。总而言之，该项目将通过促进开发新的测量工具来影响关键网络的性能、可靠性和安全性，这些工具可以发现网络优化机会、检测故障并快速识别中断在线服务的攻击。传统的测量工具和数据集虽然非常有用，但在规模和覆盖范围方面存在重大限制。测量研究人员应该利用可编程数据平面中令人兴奋的进步来分析数据包在网络中传输时的互联网流量和性能。直接在数据平面(例如网络交换机、路由器)中分析流量可以在不牺牲效率或泄露敏感用户信息的情况下实现复杂的分析，而企业网络(如大学校园)提供了在实践中使用这些可编程数据平面的绝佳机会。然而，对数据平面进行编程并不容易。现有的语言，如P4，非常低级，学习曲线非常陡峭，而且出了名的难以使用(看起来合法的程序经常无法编译)。该项目通过以Lucid的形式提供新的编程支持来解决这些痛点，Lucid是一种高级语言，旨在支持跨多个位置和设备类型的协作测量。更具体地说，研究团队正在开发面向Intel Tofino可编程交换机(通过P4)和软件服务器(通过eBPF)的编译器。使用这两种设备，研究人员将能够开发和部署一系列不同类型的分布式测量工具。研究小组还将为该语言开发一个翻译器，以便在部署之前开发和调试有趣的新研究想法。研究小组开发的基础设施还将包括一套库，这些库实现了在网络测量和支持数据隐私方面有用的关键数据结构和实用程序。为了教会社区如何使用我们的语言、图书馆、工具和基础设施，该团队将开发文档和教程。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Understanding the flow of traffic across key networks---what it is composed of and how it changes---is critical for improving modern information services. Traditionally, however, it has been difficult for researchers to develop new tools for dissecting this traffic and analyzing its characteristics, while taking care to maintain user privacy. Recently, though, the development of relatively cheap programmable switches has made it possible to develop diagnostic tools and place them directly inside the network, on the path through which traffic flows. In such a position, new tools have the potential to see all the internet traffic as it flows by, from a university campus to the broader internet, for instance, or along a corporate wide-area network or data center. Unfortunately, while it is possible to develop such tools, doing so is currently an incredibly difficult and error-prone process. To ameliorate this situation, the research team will develop Lucid, a new programming language and system that will facilitate the process of developing, debugging, and deploying network measurement tools in live programmable networks. The research team will deliver a compiler that translates high-level Lucid programs into lower-level code that execute in multiple places---directly on programmable switches, or in support, on servers connected to the network in question. In addition, the team will deliver a collection of reusable components that network measurement researchers can plug together to get started on a new idea quickly. To help teach researchers how to use the new language, the team is developing tutorials for major conferences in networking. To summarize, this project will impact the performance, reliability, and security of critical networks by facilitating the development of new measurement tools that can discover network optimization opportunities, detect failures, and rapidly recognize attacks that disrupt online services.Traditional measurement tools and datasets, while incredibly useful, have significant limitations in scale and coverage. Measurement researchers should capitalize on the exciting advances in programmable data planes to analyze Internet traffic and performance as packets traverse the network. Analyzing traffic directly in the data plane (e.g., network switches, routers) enables sophisticated analysis without sacrificing efficiency or divulging sensitive user information, and enterprise networks, such as university campuses, provide an excellent opportunity to use these programmable data planes in practice. However, programming the data plane is not easy. Existing languages, such as P4, are very low-level, have an extremely steep learning curve, and are notoriously difficult to work with (with seemingly legitimate programs often failing to compile). This project addresses these pain points by delivering new programming support in the form of Lucid, a high-level language designed to support cooperative measurement across multiple locations and device types. More specifically, the research team is developing compilers that will target both Intel Tofino programmable switches (via P4) and software servers (via eBPF). Using both kinds of devices, researchers will be able to develop and deploy a range of different kinds of distributed measurement tools. The research team will also develop an interpreter for the language so that interesting new research ideas may be developed and debugged prior to deployment. The infrastructure developed by the research team will also include a suite of libraries that implement key data structures and utilities useful in network measurement and in support of data privacy. To teach the community how to use our language, libraries, tools, and infrastructure, the team will develop documentation and tutorials.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.