INCITE: A Framework and Methodology for Edge-Based Traffic Processing and Service Inference

计算机网络的爆炸性增长，加上应用程序和工作负载的快速和不可预测的发展，使得网络服务推理和性能预测的要求越来越高，任务越来越棘手。尽管如此，终端系统必须了解内部网络流量状况和服务，以便验证、预测或增强要求苛刻的应用程序所需的性能。网络服务提供商也非常需要在不使用全局策略的情况下评估自己子系统的性能。没有特殊用途的网络支持，唯一的选择是使用基于边缘的网络流量处理间接推断动态网络特征。INCITE（边缘的互联网控制和推理工具）项目集中了来自网络、数字信号处理和应用数学领域的专家，他们的目标是仅基于主机和/或边缘路由器的边缘测量来表征网络服务。该项目融合了最近在多重分形流量建模、服务质量（QoS）测量和网络断层扫描方面的研究成果，以开发一个独特而创新的网络服务推理框架。INCITE项目将使用最新的dsp驱动网络处理器技术开发新的算法和实现，为更好地管理和理解互联网性能迈出重要的一步。我们的工作包括三个密切相关的研究重点：1。多重分形流量和路径建模：我们将开发新的、高度精确的工具，用于分析、建模和测量网络连接和端到端边缘路径的动态。我们的方法来推断路径上的竞争交叉交通负载利用创新的指数间隔探测序列，是由多重分形随机过程理论的启发。这些探测包啁啾在用探测淹没网络和获得足够丰富的统计数据以进行准确估计之间取得平衡。多类服务推断：我们将为客户开发一个框架，以基于外部和被动观察来评估网络的核心qos功能。使用流量包络理论、最大似然估计和假设检验，客户端将能够评估一套广泛的网络多类控制机制，如服务规程、链路共享规则和参数、监管参数。单播网络断层扫描：我们将开发一种新的网络断层扫描方法，该方法基于网络边缘的单播流量测量，为任意拓扑的网络提供视频链路级性能表征。基于因子图的新网络建模框架将使链路级服务参数（例如，损失，延迟和服务策略）的统计推断成为可能。我们设想的方法的一个关键优势是，它将使可扩展的实时断层扫描算法部署在主机和/或边缘路由器上。INCITE项目将为复杂的大规模网络开发网络多重分形流量处理、业务推理和链路级表征的理论基础，并导致仅基于网络边缘流量测量的计算高效和可扩展的业务推理算法。此外，通过与宽带互联网带宽的领先供应商（安然）和网络信号处理硬件的创新者（德州仪器）的合作，我们将构建所提出算法的完整原型实现，包括多重分形流量和路径建模、服务推理和网络断层扫描的模块。这一参考实现将为获得对大型网络的深入理解提供首个同类平台，并使未来网络体系结构、算法和模型的原则设计成为可能。

The explosive growth of computer networks, combined with rapid and unpredictable developments in ap-plications and workloads, has rendered network service inference and performance prediction increasingly de-manding and intractable tasks. Nonetheless, end-systems must have knowledge of internal network traffic con-ditions and servicing in order to validate, predict, or enhance performance capabilities required by demanding applications. Network service providers also have a great need for gauging the performance of their own sub-systems without recourse to global strategies. Without special-purpose network support, the only alternative is to indirectly infer dynamic network characteristics using edge-based network traffic processing. The INCITE (InterNet Control and Inference Tools at the Edge) Project focuses experts from the fields ofnetworking, digital signal processing, and applied mathematics towards the goal of characterizing network ser-vice based solely on edge-based measurement at hosts and/or edge routers. This project blends recent work in multifractal traffic modeling, quality of service (QoS) measurement, and network tomography to develop a unique and innovative framework for network service inference. The INCITE Project will develop new al-gorithms and implementations using the latest in DSP-driven network processor technology, providing a vital step towards better managing and understanding of Internet performance. Our effort consists of three closely inter-related research thrusts:1. Multifractal Traffic and Path Modeling: We will develop new, highly accurate tools for analyzing,modeling, and measuring the dynamics of network connections and end-to-end paths from the edge. Ourapproach to inferring the competing cross-traffic load on a path utilizes an innovative exponentially spacedprobing sequence that is inspired by the theory of multifractal random processes. These probing packetchirps balance the trade-off between overwhelming the network with probes and obtaining statistics richenough for accurate estimates.2. Multiclass Service Inference: We will develop a framework for clients to assess a network's core QoSfunctionalities based on external and passive observations. Using the theory of traffic envelopes, maximumlikelihood estimation, and hypothesis testing, clients will be able to assess a broad set of the network'smulti-class control mechanisms such as the service disciplines, link sharing rules and parameters, andpolicing parameters.3. Unicast Network Tomography: We will develop a novel methodology for network tomography that pro-videslink-level performance characterization of networks of arbitrary topologies based on unicast trafficmeasurements at a the network edge. A new network modeling framework based on factor graphs willenable the statistical inference of link-level service parameters (e.g., losses, delays, and service strate-gies). A key strength of our envisioned methodology is that it will enable scalable, real-time tomography algorithms deployable on hosts and/or edge routers. The INCITE Project will develop the theoretical underpinnings of network multifractal traffic processing,service inference, and link-level characterization for complex, large-scale networks, and lead to computation-ally efficient and scalable service inference algorithms based only on traffic measurement at the network edge. Moreover, in collaboration with a leading provider of broadband Internet bandwidth (Enron) and an innovator in networked signal processing hardware (Texas Instruments), we will build a complete prototype implementation of the proposed algorithms, including modules for multifractal traffic and path modeling, service inference, and network tomography. This reference implementation will provide a first-of-its-kind platform for obtaining a deep understanding of large networks and enable principled designs of future network architectures, algorithms, and models.