Online Optimization of File Transfers in High-Speed Networks
Online Optimization of File Transfers in High-Speed Networks
复制标题
DOI：
10.1145/3458817.3476208
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
M. Arifuzzaman;Engin Arslan
中科院分区：
文献类型：
--
作者：
M. Arifuzzaman;Engin Arslan
高速网络中的文件传输需要网络和I/O并行性来达到高速，然而，创建任意大量的I/O和网络线程会占用系统资源并导致公平性问题。在本文中，我们介绍了Falcon，它结合了一种新的效用函数与最先进的在线优化算法，以发现文件传输的I/O和网络并行度，可以最大限度地提高吞吐量，同时保持系统开销低，并确保竞争传输之间的公平性。我们在几个专用和生产高速网络中进行的广泛评估表明，Falcon可以在短短20秒内找到接近最佳的解决方案，并且比现有的传输应用性能高出2- 6倍。此外，与其他文件传输优化解决方案无法确保竞争传输之间的公平资源分配不同，当多个Falcon代理在其博弈论启发的效用函数的帮助下竞争网络资源时，Falcon可以保证收敛到纳什均衡。
File transfers in high-speed networks require network and I/O parallelism to reach high speeds, however, creating arbitrarily large numbers of I/O and network threads overwhelms system resources and causes fairness issues. In this paper, we introduce Falcon that combines a novel utility function with state-of-the-art online op-timization algorithms to discover the degree of I/O and network parallelism for file transfer that can maximize the throughput while keeping system overhead low and ensuring fairness among competing transfers. Our extensive evaluations in several dedicated and production high-speed networks show that Falcon can find near-optimal solution in as little as 20 seconds and outperforms existing transfer application by 2-6x. Moreover, unlike other file transfer optimization solutions that fail to ensure fair resource allocation between competing transfers, Falcon is guaranteed to converge to Nash Equilibrium when multiple Falcon agents compete for the network resources with the help of its game theory-inspired utility function.