Speeding-up reinforcement learning through abstraction and transfer learning
Speeding-up reinforcement learning through abstraction and transfer learning
复制标题
通过抽象和迁移学习加速强化学习
DOI：
10.1145/2490000/2484942/p119-koga.pdf
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Anna Helena Reali Costa
中科院分区：
文献类型：
--
作者：
Marcelo Li Koga;V. F. Silva;Fabio Gagliardi Cozman;Anna Helena Reali Costa
我们感兴趣的是以下的一般性问题：是否有可能将在学习解决问题的过程中产生的知识抽象出来，以便这种抽象可以加速学习过程？此外，是否有可能转移和重用获得的抽象知识来加速未来类似任务的学习过程？我们提出了一个框架，用于同时进行两个层次的强化学习，在学习问题的具体策略的同时学习抽象策略，这样两个策略都可以通过探索和智能体与环境的交互来改进。我们探索抽象既可以加速当前问题的最佳具体策略的学习过程，也可以将生成的抽象策略应用于新问题的学习解决方案。我们报告了在机器人导航环境中的实验，表明我们的框架在加速实际问题的策略构建和生成可用于加速新类似问题学习的抽象方面是有效的。
We are interested in the following general question: is it possible to abstract knowledge that is generated while learning the solution of a problem, so that this abstraction can accelerate the learning process? Moreover, is it possible to transfer and reuse the acquired abstract knowledge to accelerate the learning process for future similar tasks? We propose a framework for conducting simultaneously two levels of reinforcement learning, where an abstract policy is learned while learning of a concrete policy for the problem, such that both policies are refined through exploration and interaction of the agent with the environment. We explore abstraction both to accelerate the learning process for an optimal concrete policy for the current problem, and to allow the application of the generated abstract policy in learning solutions for new problems. We report experiments in a robot navigation environment that show our framework to be effective in speeding up policy construction for practical problems and in generating abstractions that can be used to accelerate learning in new similar problems.