PhD Proposal: Efficient Traffic Detection, Scheduling and Transmission in High-Speed Networks

越来越多的公司建立大型数据中心来提供在线服务，包括网络搜索、在线游戏等。为了支持这些服务，在高速网络中已经采用了多根树拓扑，其利用源主机和目的主机之间的多条路径来保证高带宽。一般来说，高速网络中的流量遵循重尾分布，即10%的流量携带大约90%的数据，而其中大约90%的流量产生10%的总需求。大多数短流是延迟敏感的。在没有事先区分的情况下调度这些数据将使它们容易经历长的排队延迟和数据包重新排序，这会增加它们的总体完成时间并降低应用程序性能。因此，在大规模高速网络中，准确地检测线速率下的重流并在不同路径上适当地调度流量以获得低流竞争时间是至关重要的。此外，需要新的传输协议来提高这种高速网络中链路资源的利用率。在这个项目中，我将设计一套健壮的流量工程解决方案来改善重流检测的准确性问题，减少流竞争时间：（1）用于高准确度流检测的数据结构：在当前的重流检测方法中，流持久性计数器在面临内存限制时被天真地替换，导致检测准确性低。我计划设计一个新的草图方案，通过概率替换方法更仔细地替换现有的流指标，同时通过记录流键（例如，五元组信息）在更新和查询操作方面保持高吞吐量。(2)具有自适应重路由粒度的灵活流调度：由于流量动态和链路故障，当前的高速网络在不同的路径下运行，这导致拓扑不对称。对于流调度，现有的流级方法将每个流映射到一个路径，并且小流级方法仅在新的小流出现时重新路由小流。这两种方案即使在对称拓扑下，由于其粗糙的切换粒度，也会导致网络资源利用率低。相比之下，细粒度机制在非对称拓扑中容易发生严重的数据包重排，导致性能下降。为了减少这种情况下的流完成时间，我将提出一种新的流调度机制，根据实时网络状态自适应调整重路由粒度。(3)流传输的精确子流调整：传统TCP的目标是在广域网中实现可靠的传输，依赖于作为拥塞信号的数据包丢失。Vanilla TCP不适用于数据中心网络，因为它会降低传输性能，特别是短流。这是因为只要没有丢包，数据包就会存储在交换机缓冲区中，这可能导致长队列，从而导致大的流完成时间。MPTCP使用几个子流来避免分组重新排序并减少流完成时间。然而，当前的MPTCP设计通常不知道子流的数量，导致带宽利用不足或频繁的超时事件。为了避免长时间的排队延迟，并为短流提供低延迟传输，同时保证长流的高吞吐量，我计划设计一种传输机制，利用深度强化学习的力量来学习如何根据高速网络中的实时网络状态灵活调整子流的数量。数据集：对于流量检测，我将使用CAIDA的匿名IP跟踪。对于流调度和传输，我将应用公共Facebook数据集。预期成果：预期成果将是提交给IEEE INFOCOM，IEEE ICNP或ACM CoNEXT的几篇研究论文。开发的代码将在Github上发布。

More and more companies build large data centers to provide online services, including web search, online gaming, etc. To support these services, multi-rooted tree topologies have been employed in high-speed networks, which utilise multiple paths between source and destination hosts to guarantee high bandwidth. In general, traffic in high-speed networks follows a heavy-tailed distribution, i.e. 10% of flows carry approximately 90% of data, while approximately 90% of them generate 10% of the overall demand. Most short flows are delay-sensitive. Scheduling these without prior discrimination will make them prone to experiencing long queuing delays and packet reordering, which increases their overall completion time and deteriorates application performance. Thus, accurately detecting heavy flows at line rate and scheduling traffic on different paths appropriately, to attain low flow competition times is crucial in large-scale high-speed networks. In addition, new transport protocols are needed to increase the utilization of link resources in such high-speed networks. In this project, I will design a set of robust traffic engineering solutions to ameliorate the heavy flow detection accuracy problem and reduce flow competition times: (1) Data structures for highly-accurate flow detection: In current heavy flow detection approaches, flow persistence counters are naively replaced when facing memory constraints, resulting in low detection accuracy. I plan to design a new sketch scheme to replace incumbent flow indicators more carefully through a probabilistic replacement approach, while maintaining high throughput in terms of update and query operations via recording flow keys (e.g. five-tuple information). (2) Flexible flow scheduling with adaptive rerouting granularity: Current high-speed networks run under diverse paths due to traffic dynamics and link failures, which results in topology asymmetries. For flow scheduling, existing flow-level approaches map each flow to one path, and flowlet-level approaches only reroute flowlets when a new flowlet emerges. Both kinds of schemes suffer from low utilization of network resources due to their coarse switching granularity even under symmetric topologies. In contrast, fine-grained mechanisms are prone to severe packet reordering in asymmetric topologies, leading to performance degradation. To decrease flow completion time in such circumstances, I will propose a new flow scheduling mechanism that adjusts the rerouting granularity adaptively according to the real-time network status. (3) Accurate subflow adjustment for flow transmission: Traditional TCP is aimed at reliable transport in wide-area networks, relying on packet loss as the congestion signal. Vanilla TCP is inappropriate for data center networks, since it deteriorates transmission performance, particularly of short flows. This is because packets will be stored in switch buffers as long as there is no packet loss, which may result in long queues and thus large flow completion times. MPTCP uses several subflows to avert packet reordering and reduce flow completion time. However, current MPTCP designs are usually unaware of the number of subflows, leading to bandwidth under-utilization or frequent timeout events. In order to avoid long queuing delays and provide low-latency transmission for short flows while guaranteeing high throughput for long ones, I plan to design a transmission mechanism that harnesses the power of deep reinforcement learning to learn how to adjust the number of subflows flexibly according to real-time network status in high-speed networks. Datasets: For flow detection, I will utilize anonymized IP traces from CAIDA. For flow scheduling and transmission, I will apply the public Facebook dataset. Expected Outputs: The expected outcome will be several research papers to be submitted to IEEE INFOCOM, IEEE ICNP or ACM CoNEXT. The code developed will be published on Github.