A Distributed Q-Learning Algorithm for Multi-Agent Team Coordination
A Distributed Q-Learning Algorithm for Multi-Agent Team Coordination
复制标题
一种用于多智能体团队协调的分布式 Q-Learning 算法
DOI：
--
复制
发表时间：
2005
期刊：
影响因子：
--
通讯作者：
Da
中科院分区：
文献类型：
--
作者：
Jing Huang;Bo Yang;Da
Q学习是一种有效的无模型强化学习算法。然而，Q-学习是集中式的，只适合于单智能体学习，而不是多智能体学习，因为在后一种情况下，状态-动作空间的大小是巨大的，并会随着智能体数量的增加而呈指数级增长。在本文中，我们提出了一个分布式Q学习算法来解决这个问题。在我们的算法中，学习最优行动策略的任务被分配给团队中的每个代理，而不是一个中央代理。为了减小多Agent团队的动作-状态空间的大小，本文引入了一种Agent团队的状态-动作空间共享策略，通过该策略，团队中的一个Agent可以使用其他Agent已经探索过的状态，而不需要花费时间再次探索这些状态。此外，我们的算法有能力分配子目标之间的代理根据环境的变化，这可以使代理团队更有效地协调。实验表明，我们的算法的效率时，它被应用到基准问题的捕食者-猎物的追求游戏，也被称为追求游戏，其中一队捕食者协调捕捉猎物。
Q-learning is an effective model-free reinforcement learning algorithm. However, Q-learning is centralized and competent only for single agent learning but not multi-agent learning because in later case the size of state-action space is huge and will grow exponentially with the number of agents increasing. In the paper we present a distributed Q-learning algorithm to solving this problem. In our algorithm, the tasks of learning optimal action policy are distributed to each agent in team but not a central agent. In order to reduce the size of action-state space of multi-agent team we introduce a state-action space sharing strategy of agent team, through which one agent in team can use the states already explored by other agents before and need not take time to explore these states again. Additionally, our algorithm has the ability to allocate sub-goals dynamically among agents according to environment changing, which can make agent team coordinate more efficiently. Experiments show the efficiency of our algorithm when it is applied to the benchmark problem of predator-prey pursuit game, also called pursuit game, in which a team of predators coordinate to capture a prey.