Lipschitz Lifelong Reinforcement Learning
Lipschitz Lifelong Reinforcement Learning
复制标题
DOI：
10.1609/aaai.v35i9.17006
复制
发表时间：
2020-01
期刊：
影响因子：
--
通讯作者：
Erwan Lecarpentier;David Abel;Kavosh Asadi;Yuu Jinnai;E. Rachelson;M. Littman
中科院分区：
文献类型：
--
作者：
Erwan Lecarpentier;David Abel;Kavosh Asadi;Yuu Jinnai;E. Rachelson;M. Littman
我们考虑当一个智能体面临一系列强化学习（RL）任务时的知识转移问题。我们引入了一个新的马尔可夫决策过程之间的度量，并建立密切的MDP有密切的最优值函数。形式上，最优值函数关于任务空间是Lipschitz连续的。这些理论结果使我们的终身RL，我们使用它来建立一个PAC-MDP算法与改进的收敛速度的值转移方法。此外，我们展示了以高概率不经历负迁移的方法。我们说明了终身RL实验的方法的好处。
We consider the problem of knowledge transfer when an agent is facing a series of Reinforcement Learning (RL) tasks. We introduce a novel metric between Markov Decision Processes and establish that close MDPs have close optimal value functions. Formally, the optimal value functions are Lipschitz continuous with respect to the tasks space. These theoretical results lead us to a value-transfer method for Lifelong RL, which we use to build a PAC-MDP algorithm with improved convergence rate. Further, we show the method to experience no negative transfer with high probability. We illustrate the benefits of the method in Lifelong RL experiments.