Collaborative Research: CNS Core: Small: Understanding Per-Hop Flow Control

本研究关注如何最好地管理数据中心网络资源的争用。数据中心是计算机行业中发展最快的部分之一，网络将数据中心中的计算机连接起来，使它们能够进行通信。正如太多的人试图同时使用道路会变得拥挤一样，当太多的应用程序试图同时发送数据时，数据中心网络也会变得拥挤。如今，大多数网络都使用端到端控制机制——当网络变得拥塞时，它会向计算机发送信号以使其减速。似乎更快的网络会有所帮助，但事实恰恰相反——网络通信的数量也在迅速增加，在反馈机制开始控制流量之前，可以发送更多的数据。这个项目（华盛顿大学和麻省理工学院的研究人员合作的项目）是为了探索一种不同的方法，其中反馈发生在网络中，在网络交换机之间一跳一跳地进行，只是为了那些发送速度太快的应用程序。数据中心拥塞控制面临的挑战包括快速增长的工作负载需求、更快的链路、较小的平均传输大小、极其突发的流量以及有限的交换机缓冲容量。现有的端到端拥塞控制系统在这些设置中远非最佳，这对于延迟敏感的应用程序尤其明显。许多数据中心运营商通过使用优先级和/或以非常低的平均利用率运行网络来补偿，但这增加了成本，而没有完全解决问题。本研究试图了解一种基于每跳流控制的数据中心网络拥塞控制替代方法的优点和局限性。该研究将(i)开发一个理论框架来量化两种不同方法之间的差异，（ii）展示在现代可编程数据中心网络交换机上的实际实现，以及（iii）理解和开发解决方案，以应对在数据中心中使用每跳流控制的工程挑战。如果成功，该研究将有助于以更低的成本在数据中心内部和跨数据中心部署新兴的延迟敏感应用程序，用于突发流量模式和正在开发的工业中新兴的非常高带宽网络。数据中心网络技术正在迅速发展，因此本研究的一个关键方面是开发材料，以帮助培训本科生和研究生应对延迟敏感应用程序对数据中心网络构成的挑战。项目网站https://www.cs.washington.edu/homes/tom/backpressure/包含所有项目论文、演示文稿、源代码、模拟、实验结果和教学材料的副本。随着项目的进展，额外的材料将放置在那里，并在项目完成后至少保留十年。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research concerns how best to manage contention for data center network resources. Data centers are among the fastest growing segment of the computer industry, and networks connect the computers in a data center to allow them to communicate. Just as roads can become congested when too many people try to use them at the same time, data center networks can become congested when too many applications try to send data at the same time. Most networks today use an end to end control mechanism - as the network becomes congested, it sends signals back to the computers to slow down. It might seem that faster networks would help, but the opposite is true - the amount of network communication is also rapidly increasing, and more data can be sent before the feedback mechanism can kick in to control traffic. This project (a collaborative project between investigators at the University of Washington and Massachusetts Institute of Technology) is to explore a different approach, where feedback occurs within the network, hop-by-hop between network switches, and just for those applications that are sending too fast.The challenges for congestion control for data centers include rapidly increasing workload demand, ever faster links, small average transfer sizes, extremely bursty traffic, and limited switch buffer capacity. Existing end-to-end congestion control systems are far from optimal in these settings, and this is particularly noticeable for latency-sensitive applications. Many data center operators compensate by using priorities and/or running their networks at very low average utilization, but this raises costs without fully solving the problem. This research attempts to understand the benefits and limits of an alternative approach to congestion control for data center networks, based on per-hop flow control. The research will (i) develop a theoretical framework to quantify the difference between the two different approaches, (ii) demonstrate a practical implementation on modern programmable data center network switches, and (iii) understand and develop solutions for the engineering challenges of using per-hop flow control in data centers.If successful, the research will help enable an emerging class of latency-sensitive applications to be deployed within and across data centers and at lower cost, for bursty traffic patterns and emerging very high bandwidth networks being developed in industry. Data center network technologies are rapidly evolving, and so a key aspect of this research is to develop materials to help train undergraduate and graduate students for the challenges that latency-sensitive applications pose for data center networks.The project website, https://www.cs.washington.edu/homes/tom/backpressure/, contains copies of all project papers, presentations, source code, simulations, experimental results, and teaching materials. Additional material will be placed there as the project progresses, and will be maintained for a minimum of ten years after the completion of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.