Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
复制标题
DOI：
--
复制
发表时间：
2019-01
期刊：
影响因子：
--
通讯作者：
Rui Wang;J. Lehman;J. Clune;Kenneth O. Stanley
中科院分区：
文献类型：
--
作者：
Rui Wang;J. Lehman;J. Clune;Kenneth O. Stanley
虽然迄今为止机器学习的历史主要涵盖了研究人员和学习其解决方案的算法提出的一系列问题，但一个重要的问题是算法是否可以在解决问题的同时生成问题本身。这样的过程实际上会建立自己多样化和不断扩展的课程，各个阶段问题的解决方案将成为该过程后期解决更具挑战性问题的垫脚石。本文介绍的配对开放式开拓者 (POET) 算法就是这样做的：它将环境挑战的生成与解决这些挑战的代理优化配对。它同时在可能的问题和解决方案的空间中探索许多不同的路径，并且至关重要的是，允许这些垫脚石解决方案在问题之间转移（如果更好的话），从而催化创新。开放式一词意味着像 POET 这样的算法具有令人着迷的潜力，可以继续无限制地创造新颖且日益复杂的功能。我们的结果表明，POET 产生了各种复杂的行为，可以解决各种环境挑战，其中许多挑战无法仅通过直接优化来解决，甚至不能通过引入直接路径课程构建控制算法来解决，以强调开放性在解决雄心勃勃的挑战中的关键作用。事实证明，将解决方案从一种环境转移到另一种环境的能力对于释放整个系统的全部潜力至关重要，这证明了偶然垫脚石的不可预测性。我们希望 POET 能够激发跨多个领域的开放式发现的新动力，像 POET 这样的算法可以通过其有趣的可能表现和解决方案开辟道路。
While the history of machine learning so far largely encompasses a series of problems posed by researchers and algorithms that learn their solutions, an important question is whether the problems themselves can be generated by the algorithm at the same time as they are being solved. Such a process would in effect build its own diverse and expanding curricula, and the solutions to problems at various stages would become stepping stones towards solving even more challenging problems later in the process. The Paired Open-Ended Trailblazer (POET) algorithm introduced in this paper does just that: it pairs the generation of environmental challenges and the optimization of agents to solve those challenges. It simultaneously explores many different paths through the space of possible problems and solutions and, critically, allows these stepping-stone solutions to transfer between problems if better, catalyzing innovation. The term open-ended signifies the intriguing potential for algorithms like POET to continue to create novel and increasingly complex capabilities without bound. Our results show that POET produces a diverse range of sophisticated behaviors that solve a wide range of environmental challenges, many of which cannot be solved by direct optimization alone, or even through a direct-path curriculum-building control algorithm introduced to highlight the critical role of open-endedness in solving ambitious challenges. The ability to transfer solutions from one environment to another proves essential to unlocking the full potential of the system as a whole, demonstrating the unpredictable nature of fortuitous stepping stones. We hope that POET will inspire a new push towards open-ended discovery across many domains, where algorithms like POET can blaze a trail through their interesting possible manifestations and solutions.