Dealing With Changes: Resilient Routing via Graph Neural Networks and Multi-Agent Deep Reinforcement Learning
Dealing With Changes: Resilient Routing via Graph Neural Networks and Multi-Agent Deep Reinforcement Learning
复制标题
DOI：
10.1109/tnsm.2023.3287936
复制
发表时间：
2023-09
影响因子：
5.3
通讯作者：
Sai Shreyas Bhavanasi;L. Pappone;Flavio Esposito
中科院分区：
文献类型：
--
作者：
Sai Shreyas Bhavanasi;L. Pappone;Flavio Esposito
计算机网络社区一直在稳步增加对机器学习的研究，以帮助解决路由、流量预测和资源管理等任务。互联网连接的传统尽力而为特性允许在竞争网络资源的多个流之间共享单个链路，而通常不考虑网络内状态。特别是，由于最近在其他应用中取得的成功，强化学习在网络管理以及最近的路由领域取得了稳定增长。然而，如果网络拓扑发生变化，通常需要重新训练以避免显着的性能损失。这种限制主要阻碍了基于强化学习的路由在实际环境中的部署。在本文中，我们将路由作为强化学习问题来处理，具有两个新颖的特点：最小化流集冲突，并构建能够在动态网络条件下进行路由而无需重新训练的强化学习策略。我们将这种方法与其他路由协议（包括多代理学习）在各种服务质量指标方面进行比较，并报告我们吸取的教训。
The computer networking community has been steadily increasing investigations into machine learning to help solve tasks such as routing, traffic prediction, and resource management. The traditional best-effort nature of Internet connections allows a single link to be shared among multiple flows competing for network resources, often without consideration of in-network states. In particular, due to the recent successes in other applications, Reinforcement Learning has seen steady growth in network management and, more recently, routing. However, if there are changes in the network topology, retraining is often required to avoid significant performance losses. This restriction has chiefly prevented the deployment of Reinforcement Learning-based routing in real environments. In this paper, we approach routing as a reinforcement learning problem with two novel twists: minimize flow set collisions, and construct a reinforcement learning policy capable of routing in dynamic network conditions without retraining. We compare this approach to other routing protocols, including multi-agent learning, with respect to various Quality-of-Service metrics, and we report our lesson learned.