Distributed Multi-Agent Deep Reinforcement Learning for Robust Coordination against Noise
Distributed Multi-Agent Deep Reinforcement Learning for Robust Coordination against Noise
复制标题
DOI：
10.1109/ijcnn55064.2022.9892253
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Yoshinari Motokawa;T. Sugawara
中科院分区：
文献类型：
--
作者：
Yoshinari Motokawa;T. Sugawara
在多智能体系统中，降噪技术是相当大的，以提高整体系统的可靠性，因为代理商需要依赖于有限的环境信息，以开发与周围的代理合作和协调的行为。然而，以前的研究往往采用集中式降噪方法，在嘈杂的多智能体环境中建立强大的和通用的协调，而分布式和分散的自治代理更合理的现实世界的应用。在本文中，我们介绍了一个分布式的注意力演员架构模型的多智能体系统（DA 3-X），使用它，我们证明了与DA 3-X的代理可以选择性地学习嘈杂的环境和行为合作。我们通过比较使用和不使用DA 3-X的学习方法来实验评估DA 3-X的有效性，并表明使用DA 3-X的代理可以比基线代理实现更好的性能。此外，我们将DA 3-X的注意力权重热图可视化，以分析决策过程和协调行为如何受到噪声的影响。
In multi-agent systems, noise reduction techniques are considerable for improving the overall system reliability as agents are required to rely on limited environmental information to develop cooperative and coordinated behaviors with the surrounding agents. However, previous studies have often applied centralized noise reduction methods to build robust and versatile coordination in noisy multi-agent environments, while distributed and decentralized autonomous agents are more plausible for real-world application. In this paper, we introduce a distributed attentional actor architecture model for a multi-agent system (DA3-X), using which we demonstrate that agents with DA3-X can selectively learn the noisy environment and behave cooperatively. We experimentally evaluate the effectiveness of DA3-X by comparing learning methods with and without DA3-X and show that agents with DA3-X can achieve better performance than baseline agents. Furthermore, we visualize heatmaps of attentional weights from the DA3-X to analyze how the decision-making process and coordinated behavior are influenced by noise.