NeTS: Small: Beating the Odds in Traffic Measurements/Detection with Optimal Online Learning and Adaptive Policies

分析数据包跟踪是理解和设计Internet主干的关键工具。然而，考虑到主干速度不断提高，达到100Gbps，始终监控单个流量是令人望而却步的。该项目开发了最佳的在线学习和适应策略，用于在硬资源限制(例如，路由器的CPU或内存有限)和动态网络/流量条件下进行准确的流量采样、推理和检测。基于多臂强盗、群体测试和压缩传感的理论和技术，将通过利用所研究的特定测量应用的独特结构来开发最优或接近最优的解决方案。解决的挑战包括从具有重尾分布和长相关性的观测中学习，处理稀疏和/或不完美的观测，以及涉及多个监控器和决策点的分布式学习策略。如果成功，本研究将为软件定义网络(SDN)范式下灵活的流量测量基础设施提供基本设计原则。基于学习过程的可重构测量可以在使用诸如OpenFlow等SDN API的商用路由器/交换机中实现，从而带来新服务的潜在开发。该项目研究网络和随机学习/优化的交叉问题，在团队环境中为研究生和本科生提供跨学科培训。

A key tool for understanding and engineering Internet backbone is the analysis of packet traces. However, given the increasing backbone speed towards 100Gbps, it is prohibitive to monitor individual flows at all times. This project develops optimal online learning and adaptation strategies for accurate traffic sampling, inference, and detection under hard resource constraints (e.g., limited CPU or memory at routers) and dynamic network/traffic conditions. Based on theories and techniques in multi-arm bandits, group testing, and compressed sensing, optimal or near-optimal solutions will be developed by exploiting the unique structures of the specific measurement application under study. Challenges addressed include learning from observations with heavy-tailed distributions and long-range dependencies, coping with sparse and/or imperfect observations, and distributed learning strategies that involve multiple monitors and decision points.If successful, this research will provide fundamental design principles for a flexible traffic measurement infrastructure under the software-defined networking (SDN) paradigm. Reconfigurable measurements based on a learning process can be realized in commodity router/switches using SDN APIs such as OpenFlow, leading to potential development of new services. As this project examines problems at the intersection of networking and stochastic learning/optimization, it provides interdisciplinary training to graduate and undergraduate students in a team environment.