Hierarchical Reinforcement Learning with Deep Nested Agents
Hierarchical Reinforcement Learning with Deep Nested Agents
复制标题
使用深层嵌套代理的分层强化学习
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Peng Wei
中科院分区：
文献类型：
--
作者：
Marc Brittain;Peng Wei
深度层次强化学习近年来获得了很多关注，因为它能够在非层次框架无法学习有用策略的具有挑战性的环境中产生最先进的结果。然而，随着问题域变得越来越复杂，深度分层强化学习可能变得低效，导致更长的收敛时间和较差的性能。我们引入了深度嵌套代理框架，这是深度分层强化学习的一种变体，其中来自主代理的信息通过将这些信息合并到嵌套代理的状态中来传播到低层嵌套代理。我们通过将深度嵌套代理框架应用于Minecraft中的三个场景，并与深度非分层单代理框架以及深度分层框架进行比较，来展示深度嵌套代理框架的有效性和性能。
Deep hierarchical reinforcement learning has gained a lot of attention in recent years due to its ability to produce state-of-the-art results in challenging environments where non-hierarchical frameworks fail to learn useful policies. However, as problem domains become more complex, deep hierarchical reinforcement learning can become inefficient, leading to longer convergence times and poor performance. We introduce the Deep Nested Agent framework, which is a variant of deep hierarchical reinforcement learning where information from the main agent is propagated to the low level $nested$ agent by incorporating this information into the nested agent's state. We demonstrate the effectiveness and performance of the Deep Nested Agent framework by applying it to three scenarios in Minecraft with comparisons to a deep non-hierarchical single agent framework, as well as, a deep hierarchical framework.