Deep Deterministic Policy Gradients with Transfer Learning Framework in StarCraft Micromanagement
Deep Deterministic Policy Gradients with Transfer Learning Framework in StarCraft Micromanagement
复制标题
DOI：
10.1109/eit.2019.8833742
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Dong Xie;Xiangnan Zhong
中科院分区：
文献类型：
--
作者：
Dong Xie;Xiangnan Zhong
本文提出了一种基于深度确定性策略梯度(DDPG)技术的实时策略游戏《星际争霸》中的智能多智能体方法。建立了一个行动者网络和一个批评者网络，分别估计最优控制行为和相应的值函数。根据智能体自身情况和敌人信息设计了一个特殊的奖励函数，帮助智能体在博弈中进行智能控制。此外，为了加快学习过程，将迁移学习技术融入到训练过程中。具体地说，特工最初在一个简单的任务中接受培训，学习战斗的基本概念，如绕道移动、避免和加入攻击。然后，我们将这一经验转移到具有复杂和困难场景的目标任务上。实验结果表明，本文提出的基于转移学习的算法具有较好的性能。
This paper proposes an intelligent multi-agent approach in a real-time strategy game, StarCraft, based on the deep deterministic policy gradients (DDPG) techniques. An actor and a critic network are established to estimate the optimal control actions and corresponding value functions, respectively. A special reward function is designed based on the agents’ own condition and enemies’ information to help agents make intelligent control in the game. Furthermore, in order to accelerate the learning process, the transfer learning techniques are integrated into the training process. Specifically, the agents are trained initially in a simple task to learn the basic concept for the combat, such as detouring moving, avoiding and joining attacking. Then, we transfer this experience to the target task with a complex and difficult scenario. From the experiment, it is shown that our proposed algorithm with transfer learning can achieve better performance.