INCITE: A Framework and Methodology for Edge-Based Traffic Processing and Service Inference

计算机网络的爆炸性增长，加之应用和工作负载的快速和不可预测的发展，使得网络服务推理和性能预测变得越来越繁重和困难。尽管如此，终端系统必须了解内部网络流量状况和服务，以便验证、预测或增强要求苛刻的应用程序所需的性能。网络服务提供商也非常需要衡量他们自己的子系统的性能，而不需要求助于全球战略。如果没有特殊用途的网络支持，唯一的选择是使用基于边缘的网络流量处理间接推断动态网络特征。INCITE(边缘的互联网控制和推理工具)项目将来自网络、数字信号处理和应用数学领域的专家聚集在一起，目标是完全基于主机和/或边缘路由器上的基于边缘的测量来表征网络服务。该项目融合了最近在多重分形流量建模、服务质量(Qos)测量和网络层析成像方面的工作，以开发一个独特的创新网络服务推理框架。INCITE项目将使用最新的数字信号处理器驱动的网络处理器技术开发新的算法和实现，为更好地管理和了解互联网性能迈出重要的一步。我们的努力包括三个密切相关的研究推动力：1.多分形流量和路径建模：我们将开发新的、高精度的工具，从边缘分析、建模和测量网络连接和端到端路径的动态。我们的方法是利用一种创新的指数间隔探测序列来推断路径上的竞争交叉流量负载，该序列受到多重分形随机过程理论的启发。这些探测数据包啁啾平衡了用探测压倒网络和获得丰富的统计数据以进行准确估计之间的权衡。多等级服务推断：我们将为客户端开发一个框架，以基于外部和被动观察来评估网络的核心QOS功能。使用流量包络理论、最大似然估计和假设检验，客户将能够评估网络的多类别控制机制的广泛集合，例如服务规则、链路共享规则和参数以及监管参数。单播网络层析成像：我们将开发一种新的网络层析成像方法，该方法基于网络边缘的单播流量测量为任意拓扑的网络提供链路级性能表征。一种新的基于因子图的网络建模框架将能够对链路级服务参数(例如，丢失、延迟和服务策略)进行统计推断。我们设想的方法的一个关键优势是，它将使可扩展的实时层析成像算法能够部署在主机和/或边缘路由器上。INCITE项目将为复杂、大规模网络发展网络多重分形流量处理、服务推理和链路级表征的理论基础，并导致仅基于网络边缘流量测量的计算高效和可扩展的服务推理算法。此外，我们将与宽带互联网带宽的领先提供商(安然)和网络信号处理硬件的创新者(德州仪器)合作，构建所建议算法的完整原型实现，包括用于多重分形流量和路径建模、服务推理和网络断层扫描的模块。此参考实施将提供首个此类平台，帮助您深入了解大型网络，并对未来的网络架构、算法和模型进行原则性设计。

The explosive growth of computer networks, combined with rapid and unpredictable developments in ap-plications and workloads, has rendered network service inference and performance prediction increasingly de-manding and intractable tasks. Nonetheless, end-systems must have knowledge of internal network traffic con-ditions and servicing in order to validate, predict, or enhance performance capabilities required by demanding applications. Network service providers also have a great need for gauging the performance of their own sub-systems without recourse to global strategies. Without special-purpose network support, the only alternative is to indirectly infer dynamic network characteristics using edge-based network traffic processing. The INCITE (InterNet Control and Inference Tools at the Edge) Project focuses experts from the fields ofnetworking, digital signal processing, and applied mathematics towards the goal of characterizing network ser-vice based solely on edge-based measurement at hosts and/or edge routers. This project blends recent work in multifractal traffic modeling, quality of service (QoS) measurement, and network tomography to develop a unique and innovative framework for network service inference. The INCITE Project will develop new al-gorithms and implementations using the latest in DSP-driven network processor technology, providing a vital step towards better managing and understanding of Internet performance. Our effort consists of three closely inter-related research thrusts:1. Multifractal Traffic and Path Modeling: We will develop new, highly accurate tools for analyzing,modeling, and measuring the dynamics of network connections and end-to-end paths from the edge. Ourapproach to inferring the competing cross-traffic load on a path utilizes an innovative exponentially spacedprobing sequence that is inspired by the theory of multifractal random processes. These probing packetchirps balance the trade-off between overwhelming the network with probes and obtaining statistics richenough for accurate estimates.2. Multiclass Service Inference: We will develop a framework for clients to assess a network's core QoSfunctionalities based on external and passive observations. Using the theory of traffic envelopes, maximumlikelihood estimation, and hypothesis testing, clients will be able to assess a broad set of the network'smulti-class control mechanisms such as the service disciplines, link sharing rules and parameters, andpolicing parameters.3. Unicast Network Tomography: We will develop a novel methodology for network tomography that pro-videslink-level performance characterization of networks of arbitrary topologies based on unicast trafficmeasurements at a the network edge. A new network modeling framework based on factor graphs willenable the statistical inference of link-level service parameters (e.g., losses, delays, and service strate-gies). A key strength of our envisioned methodology is that it will enable scalable, real-time tomography algorithms deployable on hosts and/or edge routers. The INCITE Project will develop the theoretical underpinnings of network multifractal traffic processing,service inference, and link-level characterization for complex, large-scale networks, and lead to computation-ally efficient and scalable service inference algorithms based only on traffic measurement at the network edge. Moreover, in collaboration with a leading provider of broadband Internet bandwidth (Enron) and an innovator in networked signal processing hardware (Texas Instruments), we will build a complete prototype implementation of the proposed algorithms, including modules for multifractal traffic and path modeling, service inference, and network tomography. This reference implementation will provide a first-of-its-kind platform for obtaining a deep understanding of large networks and enable principled designs of future network architectures, algorithms, and models.