Scalability Bottlenecks in Multi-Agent Reinforcement Learning Systems
Scalability Bottlenecks in Multi-Agent Reinforcement Learning Systems
复制标题
DOI：
10.48550/arxiv.2302.05007
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Kailash Gogineni;Peng Wei;Tian Lan;Guru Venkataramani
中科院分区：
文献类型：
--
作者：
Kailash Gogineni;Peng Wei;Tian Lan;Guru Venkataramani
多智能体强化学习(MAIL)是一个很有前途的研究领域，它可以对多个自主决策智能体进行建模和控制。在在线训练过程中，MAIL算法涉及性能密集型计算，例如来自属于多个代理的大型观察行动空间的探索和开发阶段。在本文中，我们试图描述几类流行的Marl算法在其训练阶段中的可伸缩性瓶颈。我们的实验结果揭示了对限制可伸缩性的Marl算法的关键模块的新见解，并概述了可能有助于解决这些性能问题的潜在策略。
Multi-Agent Reinforcement Learning (MARL) is a promising area of research that can model and control multiple, autonomous decision-making agents. During online training, MARL algorithms involve performance-intensive computations such as exploration and exploitation phases originating from large observation-action space belonging to multiple agents. In this article, we seek to characterize the scalability bottlenecks in several popular classes of MARL algorithms during their training phases. Our experimental results reveal new insights into the key modules of MARL algorithms that limit the scalability, and outline potential strategies that may help address these performance issues.