IMR: MT: Tools for Programming Distributed Data-plane Measurements

IMR：MT：分布式数据平面测量编程工具

• 批准号: 2223515
• 负责人: David Walker
• 金额: $ 60万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2223515&HistoricalAwards=false
• 关键词: IMR; MT; Tools; Programming; Distributed

Understanding the flow of traffic across key networks---what it is composed of and how it changes---is critical for improving modern information services. Traditionally, however, it has been difficult for researchers to develop new tools for dissecting this traffic and analyzing its characteristics, while taking care to maintain user privacy. Recently, though, the development of relatively cheap programmable switches has made it possible to develop diagnostic tools and place them directly inside the network, on the path through which traffic flows. In such a position, new tools have the potential to see all the internet traffic as it flows by, from a university campus to the broader internet, for instance, or along a corporate wide-area network or data center. Unfortunately, while it is possible to develop such tools, doing so is currently an incredibly difficult and error-prone process. To ameliorate this situation, the research team will develop Lucid, a new programming language and system that will facilitate the process of developing, debugging, and deploying network measurement tools in live programmable networks. The research team will deliver a compiler that translates high-level Lucid programs into lower-level code that execute in multiple places---directly on programmable switches, or in support, on servers connected to the network in question. In addition, the team will deliver a collection of reusable components that network measurement researchers can plug together to get started on a new idea quickly. To help teach researchers how to use the new language, the team is developing tutorials for major conferences in networking. To summarize, this project will impact the performance, reliability, and security of critical networks by facilitating the development of new measurement tools that can discover network optimization opportunities, detect failures, and rapidly recognize attacks that disrupt online services.Traditional measurement tools and datasets, while incredibly useful, have significant limitations in scale and coverage. Measurement researchers should capitalize on the exciting advances in programmable data planes to analyze Internet traffic and performance as packets traverse the network. Analyzing traffic directly in the data plane (e.g., network switches, routers) enables sophisticated analysis without sacrificing efficiency or divulging sensitive user information, and enterprise networks, such as university campuses, provide an excellent opportunity to use these programmable data planes in practice. However, programming the data plane is not easy. Existing languages, such as P4, are very low-level, have an extremely steep learning curve, and are notoriously difficult to work with (with seemingly legitimate programs often failing to compile). This project addresses these pain points by delivering new programming support in the form of Lucid, a high-level language designed to support cooperative measurement across multiple locations and device types. More specifically, the research team is developing compilers that will target both Intel Tofino programmable switches (via P4) and software servers (via eBPF). Using both kinds of devices, researchers will be able to develop and deploy a range of different kinds of distributed measurement tools. The research team will also develop an interpreter for the language so that interesting new research ideas may be developed and debugged prior to deployment. The infrastructure developed by the research team will also include a suite of libraries that implement key data structures and utilities useful in network measurement and in support of data privacy. To teach the community how to use our language, libraries, tools, and infrastructure, the team will develop documentation and tutorials.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

了解跨关键网络的流量流---它的组成以及它如何变化 - 对于改善现代信息服务至关重要。但是，传统上，研究人员很难开发新工具来剖析此流量并分析其特征，同时谨慎地维护用户隐私。 但是，最近，开发相对便宜的可编程开关已使开发诊断工具并将其直接放在网络中，并将其直接放在流量流过的路径上。在这样的职位上，新工具有可能看到所有互联网流量，例如，从大学校园到更广泛的互联网，例如，沿着公司广阔的地区或数据中心。 不幸的是，尽管可以开发此类工具，但目前这样做是一个非常困难且容易出错的过程。为了改善这种情况，研究团队将开发Lucid，这是一种新的编程语言和系统，将促进在可编程网络中开发，调试和部署网络测量工具的过程。研究团队将提供一个编译器，该编译器将高级清醒程序转换为低级代码，这些代码直接在可编程交换机或支持的服务器上连接到相关网络的服务器上直接在多个位置执行。此外，团队将提供一系列可重复使用的组件，网络测量研究人员可以插入这些组件，以快速开始一个新想法。 为了帮助研究人员如何使用新语言，该团队正在为网络中的主要会议开发教程。 总而言之，该项目将通过促进新的测量工具的开发来影响关键网络的性能，可靠性和安全性，这些工具可以发现网络优化机会，检测故障并迅速识别破坏在线服务的攻击。传统测量工具和数据集，同时在规模和覆盖面上具有很大的限制，同时具有很大的限制。测量研究人员应利用可编程数据平面的激动人心的进步，以分析互联网流量和绩效，因为数据包遍及网络。直接分析数据平面中的流量（例如，网络交换机，路由器）可以实现复杂的分析，而无需牺牲效率或泄露敏感的用户信息，而企业网络（例如大学校园）为实践中使用这些可编程数据平面提供了绝佳的机会。但是，编程数据平面并不容易。现有语言（例如P4）非常低级，学习曲线非常陡峭，众所周知，很难使用（看似合法的程序通常无法编译）。该项目通过以Lucid的形式提供新的编程支持来解决这些痛点，Lucid是一种高级语言，旨在支持多个位置和设备类型的合作测量。 更具体地说，研究团队正在开发编译器，这些编译器将针对Intel Tofino可编程开关（通过P4）和软件服务器（通过EBPF）。 使用这两种设备，研究人员将能够开发和部署各种不同类型的分布式测量工具。 研究团队还将为该语言开发口译员，因此在部署之前可以开发和调试有趣的新研究思想。 研究团队开发的基础架构还将包括一组库，这些库实现了对网络测量和支持数据隐私有用的关键数据结构和公用事业。 为了教社区如何使用我们的语言，图书馆，工具和基础设施，团队将开发文档和教程。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的评估评估标准的评估来支持的。

项目成果

OrbWeaver: Using IDLE Cycles in Programmable Networks for Opportunistic Coordination

OrbWeaver：在可编程网络中使用 IDLE 循环进行机会协调

• 发表时间: 2022
• 期刊: Proceedings of the 19th USENIX Symposium on Networked Systems Design and Implementation
• 作者: Yu, Liangcheng;Sonchack, John;Liu, Vincent
• 通讯作者: Liu, Vincent