Learning to Cooperate using Hierarchical Reinforcement Learning
Learning to Cooperate using Hierarchical Reinforcement Learning
复制标题
使用分层强化学习学习合作
DOI：
--
复制
发表时间：
2006
期刊：
影响因子：
--
通讯作者：
S. Mahadevan
中科院分区：
文献类型：
--
作者：
M. Ghavamzadeh;S. Mahadevan
在本文中，我们研究了使用分层强化学习（HRL）来加速协作多智能体任务的获取。介绍了一种分层多智能体强化学习框架，提出了一种分层多智能体强化学习算法——协作式强化学习算法。我们的方法的基本属性是，层次结构的使用允许代理通过在子任务级别共享信息来更快地学习协调，而不是试图在原始动作级别学习协调。以四智能体自动导引车辆（AGV）调度问题为研究对象，研究了协同HRL算法的性能。本文还讨论了自主代理之间的理性沟通行为问题。目标是让代理学习行动和通信策略，这些策略共同优化给定通信成本的任务。我们扩展了我们的多智能体HRL框架，使其包含通信决策，并提出了一种称为COM-Cooperative HRL的协作多智能体HRL算法。我们通过一个多智能体滑行问题证明了该算法的效率以及通信代价与学习到的通信策略之间的关系。
In this paper, we investigate the use of hierarchical reinforcement learning (HRL) to speed up the acquisition of cooperative multi-agent tasks. We introduce a hierarchical multi-agent RL framework, and present a hierarchical multiagent RL algorithm called Cooperative HRL. The fundamental property of our approach is that the use of hierarchy allows agents to learn coordination faster by sharing information at the level of subtasks, rather than attempting to learn coordination at the level of primitive actions. We study the performance of the Cooperative HRL algorithm using a fouragent automated guided vehicle (AGV) scheduling problem. We also address the issue of rational communication behavior among autonomous agents in this paper. The goal is for agents to learn both action and communication policies that together optimize the task given a communication cost. We extend our multi-agent HRL framework to include communication decisions and present a cooperative multi-agent HRL algorithm called COM-Cooperative HRL. We demonstrate the eciency of this algorithm as well as the relation between the communication cost and the learned communication policy using a multi-agent taxi problem.