Gentle flow control: avoiding deadlock in lossless networks
Gentle flow control: avoiding deadlock in lossless networks
复制标题
DOI：
10.1145/3341302.3342065
复制
发表时间：
2019-08
期刊：
影响因子：
--
通讯作者：
Kun Qian;Wenxue Cheng;Tong Zhang;Fengyuan Ren
中科院分区：
文献类型：
--
作者：
Kun Qian;Wenxue Cheng;Tong Zhang;Fengyuan Ren
分布式系统中的许多应用程序依赖底层无损网络来实现所需的性能。现有的无损网络解决方案提出了不同的逐跳流量控制来保证零丢包。然而，另一个关键问题也随之出现，即网络死锁。一旦系统陷入死锁，很大一部分网络就会瘫痪。现有的死锁避免解决方案将所有注意力集中在打破循环缓冲区依赖性以消除循环等待（死锁的必要条件之一）。然而，这些解决方案对网络配置施加了许多限制，并且对性能产生了副作用。在这项工作中，我们探索了解决网络死锁的全新视角：避免保持和等待情况（另一个必要条件）。实验观察表明，现有流量控制方案驱动的上行端口频繁暂停是导致hold and wait的根本原因。我们提出温和流控制（GFC）来以细粒度控制端口速率，因此即使存在循环缓冲区依赖性，所有端口也可以保持数据包流动，并证明GFC理论上可以消除死锁。我们还介绍了如何在主流无损网络（融合增强以太网和InfiniBand）中进行适度修改来实现GFC。此外，测试台实验和数据包级仿真验证了GFC可以有效避免死锁，并且带宽占用率小于0.5%。
Many applications in distributed systems rely on underlying lossless networks to achieve required performance. Existing lossless network solutions propose different hop-by-hop flow controls to guarantee zero packet loss. However, another crucial problem called network deadlock occurs concomitantly. Once the system traps in a deadlock, a large part of network would be disabled. Existing deadlock avoidance solutions focus all their attentions on breaking the cyclic buffer dependency to eliminate circular wait (one necessary condition of deadlock). These solutions, however, impose many restrictions on network configurations and side-effects on performance. In this work, we explore a brand-new perspective to solve network deadlock: avoiding hold and wait situation (another necessary condition). Experimental observations tell that frequent pause on upstream ports driven by existing flow control schemes is the root cause of hold and wait. We propose Gentle Flow Control (GFC) to manipulate the port rate at a fine granularity, so all ports can keep packets flowing even cyclic buffer dependency exists, and prove GFC can eliminate deadlock theoretically. We also present how to implement GFC in mainstream lossless networks (Converged Enhanced Ethernet and InfiniBand) with moderate modifications. Furthermore, testbed experiments and packet-level simulations validate GFC can efficiently avoid deadlock and introduce less than 0.5% of bandwidth occupation.