Prioritized Experience Replay
Prioritized Experience Replay
复制标题
DOI：
--
复制
发表时间：
2015-11
期刊：
影响因子：
--
通讯作者：
T. Schaul;John Quan;Ioannis Antonoglou;David Silver
中科院分区：
文献类型：
--
作者：
T. Schaul;John Quan;Ioannis Antonoglou;David Silver
经验回放让在线强化学习智能体能够记住并复用过去的经验。在先前的研究中，经验转换是从回放记忆中均匀采样的。然而，这种方法只是按照原始经历的相同频率回放转换，而不考虑其重要性。在本文中，我们开发了一个对经验进行优先级排序的框架，以便更频繁地回放重要的转换，从而更高效地学习。我们在深度Q网络（DQN）中使用了优先经验回放，DQN是一种强化学习算法，在许多雅达利游戏中都达到了人类水平的性能。具有优先经验回放的DQN达到了新的最先进水平，在49款游戏中的41款上优于具有均匀回放的DQN。
Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory. However, this approach simply replays transitions at the same frequency that they were originally experienced, regardless of their significance. In this paper we develop a framework for prioritizing experience, so as to replay important transitions more frequently, and therefore learn more efficiently. We use prioritized experience replay in Deep Q-Networks (DQN), a reinforcement learning algorithm that achieved human-level performance across many Atari games. DQN with prioritized experience replay achieves a new state-of-the-art, outperforming DQN with uniform replay on 41 out of 49 games.