Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning
Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning
复制标题
通过分层强化学习的自我对弈学习目标嵌入
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
R. Fergus
中科院分区：
文献类型：
--
作者：
Sainbayar Sukhbaatar;Emily L. Denton;Arthur Szlam;R. Fergus
在分层强化学习中，一个主要的挑战是确定适当的低级策略。我们提出了一种基于Sukhbaatar et al.（2018）的非对称自我游戏的无监督学习方案，该方案自动学习环境中子目标的良好表示以及可以执行它们的低级策略。然后，高级策略可以通过生成一系列连续的子目标向量来指导较低的策略。我们使用Mazebase和Mujoco环境评估我们的模型，包括具有挑战性的AntGather任务。子目标嵌入的可视化显示了环境中任务的逻辑分解。量化，我们的方法获得了令人信服的性能增益超过非层次的方法。
In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can execute them. A high-level policy can then direct the lower one by generating a sequence of continuous sub-goal vectors. We evaluate our model using Mazebase and Mujoco environments, including the challenging AntGather task. Visualizations of the sub-goal embeddings reveal a logical decomposition of tasks within the environment. Quantitatively, our approach obtains compelling performance gains over non-hierarchical approaches.