Learning Independently from Causality in Multi-Agent Environments
Learning Independently from Causality in Multi-Agent Environments
复制标题
DOI：
10.5220/0011747900003411
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Rafael Pina;V. D. Silva;Corentin Artaud
中科院分区：
文献类型：
--
作者：
Rafael Pina;V. D. Silva;Corentin Artaud
多智能体强化学习（MARL）是机器学习领域中一个越来越受关注的领域。尽管取得了显著的进展，但仍有一些问题需要调查。懒惰代理病理是MARL中一个著名的问题，它指的是MARL团队中的一些代理不为共同目标做出贡献，让团队成员做所有的工作。在这项工作中，我们旨在从基于因果关系的角度来研究这个问题。我们打算在MARL和因果关系领域之间建立一座桥梁，并讨论这种联系的有用性。我们研究了一个完全分散的MARL设置，其中智能体需要学习合作策略，并表明个人观察和团队奖励之间存在因果关系。所进行的实验表明，这种关系如何用于改进MARL中的独立代理，不仅可以提高团队的表现，还可以提高个体代理的智能行为。
Multi-Agent Reinforcement Learning (MARL) comprises an area of growing interest in the field of machine learning. Despite notable advances, there are still problems that require investigation. The lazy agent pathology is a famous problem in MARL that denotes the event when some of the agents in a MARL team do not contribute to the common goal, letting the teammates do all the work. In this work, we aim to investigate this problem from a causality-based perspective. We intend to create the bridge between the fields of MARL and causality and argue about the usefulness of this link. We study a fully decentralised MARL setup where agents need to learn cooperation strategies and show that there is a causal relation between individual observations and the team reward. The experiments carried show how this relation can be used to improve independent agents in MARL, resulting not only on better performances as a team but also on the rise of more intelligent behaviours on individual agents.