Deep Surrogate Assisted Generation of Environments
Deep Surrogate Assisted Generation of Environments
复制标题
DOI：
10.48550/arxiv.2206.04199
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Varun Bhatt;Bryon Tjanaka;Matthew C. Fontaine;S. Nikolaidis
中科院分区：
文献类型：
--
作者：
Varun Bhatt;Bryon Tjanaka;Matthew C. Fontaine;S. Nikolaidis
强化学习（RL）的最新进展已经开始产生能够解决复杂环境分布的通用代理。这些代理通常在固定的人类创作环境中进行测试。另一方面，质量多样性（QD）优化已被证明是一个有效的组成部分，环境生成算法，它可以生成高质量的环境，是不同的，在所产生的代理行为的集合。然而，这些算法需要潜在的昂贵的模拟代理新生成的环境。我们提出了深度代理辅助环境生成（DSAGE），这是一种样本高效的QD环境生成算法，它维护了一个深度代理模型，用于预测新环境中的代理行为。在两个基准领域的结果表明，DSAGE显着优于现有的QD环境生成算法，发现集合的环境，引起不同的行为，一个国家的最先进的RL代理和规划代理。我们的源代码和视频可以在https://dsagepaper.github.io/上找到。
Recent progress in reinforcement learning (RL) has started producing generally capable agents that can solve a distribution of complex environments. These agents are typically tested on fixed, human-authored environments. On the other hand, quality diversity (QD) optimization has been proven to be an effective component of environment generation algorithms, which can generate collections of high-quality environments that are diverse in the resulting agent behaviors. However, these algorithms require potentially expensive simulations of agents on newly generated environments. We propose Deep Surrogate Assisted Generation of Environments (DSAGE), a sample-efficient QD environment generation algorithm that maintains a deep surrogate model for predicting agent behaviors in new environments. Results in two benchmark domains show that DSAGE significantly outperforms existing QD environment generation algorithms in discovering collections of environments that elicit diverse behaviors of a state-of-the-art RL agent and a planning agent. Our source code and videos are available at https://dsagepaper.github.io/.