权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

NeTS: Medium: From Packets to Insights: Programmable Streaming Analytics for Networks

NeTS：中：从数据包到见解：网络的可编程流分析

基本信息

批准号：
1704077
负责人：
Jennifer Rexford
金额：
$ 120万
依托单位：
Princeton University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-07-01 至 2022-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1704077&HistoricalAwards=false
关键词：
NeTS Medium Packets Insights Programmable

项目摘要

The ability to monitor Internet traffic on our communications networks is of critical importance to our nation's economic prosperity and national security. For communications networks to run well, however, network operators must be able to manage them: they must be able to detect, diagnose, and fix problems that degrade the performance of the applications we use, and they must be able to detect and mitigate attacks against the infrastructure. To ensure that computer networks are secure and perform well, network operators need to gather measurements to detect attack traffic, diagnose performance problems, identify flaky equipment, drive traffic-engineering decisions, and more. Although network devices provide reasonable mechanisms for monitoring the control plane -- that part of the network that is responsible for routing packets/information through the network. Tools and mechanisms for monitoring the flow of network traffic remain primitive (e.g., ping and traceroute for active measurement, Netflow and sFlow for passive measurement). These measurements provide coarse statistics about network traffic or conditions, but they provide at once both too little information (because they obscure important details about the flows, such as packet timings, queue sizes, and loss rates) and too much information (because, for any particular question about performance or security, the operator needs detailed information about a few flows, as opposed to coarse information about all of them). This project aims to develop measurements that are "just right" for each of the above tasks and to design a data-analytics platform for querying the data that can be used to diagnose and mitigate network problems. Two technological trends enable fundamentally new paradigms for network measurement. The first trend is the rise of programmable network hardware -- including reconfigurable application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and network processors -- that are fast and inexpensive enough for use in commodity switches, and also programmable in target-independent languages like P4. The second trend is the emergence of scalable streaming analytics platforms, such as Spark Streaming and Apache Storm. These platforms make it possible to express queries based on streams of tuples and efficiently filter and aggregate the data. Using the programmable functionality from switches, one can define the types of tuples that a switch exports, and even perform simple computations over the tuples directly in the data plane. Given input tuple streams from one or more switches, the stream processor can compute the answer to a high-level query. This project is developing a streaming analytics framework that addresses these challenges. The researchers will develop a query language with familiar programming paradigms from existing streaming analytics platforms, which they will extend to support domain-specific primitives. They are also developing a runtime system that partitions this query across the stream processor and the switches in the data plane. Queries will entail network-wide aggregation, iterative "drill down" capabilities, and joins with external data sources (e.g., routing, application identification). The researchers are evaluating the feasibility and usability of this platform in the context of a wide-range of security and performance diagnosis queries that arise in operational networks.

监控通信网络上的互联网流量的能力对于我们国家的经济繁荣和国家安全至关重要。然而，为了使通信网络运行良好，网络运营商必须能够对其进行管理：他们必须能够检测、诊断和修复降低我们使用的应用程序性能的问题，并且他们必须能够检测和减轻针对基础设施的攻击。为了确保计算机网络安全且性能良好，网络运营商需要收集测量数据来检测攻击流量、诊断性能问题、识别不稳定设备、推动流量工程决策等。尽管网络设备提供了合理的机制来监控控制平面——网络的一部分，负责通过网络路由数据包/信息。用于监控网络流量的工具和机制仍然很原始（例如，用于主动测量的 ping 和 Traceroute，用于被动测量的 Netflow 和 sFlow）。这些测量提供了有关网络流量或状况的粗略统计数据，但它们同时提供的信息太少（因为它们掩盖了有关流的重要细节，例如数据包计时、队列大小和丢失率），而且信息太多（因为，对于有关性能或安全性的任何特定问题，操作员需要有关少数流的详细信息，而不是有关所有流的粗略信息）。该项目旨在为上述每项任务开发“恰到好处”的测量方法，并设计一个数据分析平台来查询可用于诊断和缓解网络问题的数据。两种技术趋势为网络测量带来了全新的范例。第一个趋势是可编程网络硬件的兴起，包括可重新配置的专用集成电路 (ASIC)、现场可编程门阵列 (FPGA) 和网络处理器，这些硬件速度快、成本低，足以在商品交换机中使用，并且还可以使用 P4 等与目标无关的语言进行编程。第二个趋势是可扩展的流分析平台的出现，例如 Spark Streaming 和 Apache Storm。这些平台使得基于元组流表达查询并有效过滤和聚合数据成为可能。使用交换机的可编程功能，可以定义交换机导出的元组类型，甚至可以直接在数据平面中对元组执行简单的计算。给定来自一个或多个交换机的输入元组流，流处理器可以计算高级查询的答案。该项目正在开发一个流分析框架来应对这些挑战。研究人员将开发一种具有来自现有流分析平台的熟悉编程范例的查询语言，他们将扩展该语言以支持特定于领域的原语。他们还在开发一个运行时系统，该系统可以跨流处理器和数据平面中的交换机划分此查询。查询将需要网络范围内的聚合、迭代“向下钻取”功能以及与外部数据源（例如路由、应用程序识别）的连接。研究人员正在运营网络中出现的各种安全和性能诊断查询的背景下评估该平台的可行性和可用性。