Effective Reuse Method for Transfer Learning in Actor-critic
Effective Reuse Method for Transfer Learning in Actor-critic
复制标题
Actor-critic 中迁移学习的有效重用方法
DOI：
10.14864/softscis.2010.0.137.0
复制
发表时间：
2010
期刊：
影响因子：
--
通讯作者：
S. Tsuruoka
中科院分区：
文献类型：
--
作者：
Toshiaki Takano;H. Takase;H. Kawanaka;S. Tsuruoka
本文旨在通过迁移学习来加速Actor-Critic方法的学习过程，Actor-Critic方法是主要的强化学习算法之一。强化学习允许智能体自主地完成目标任务。迁移学习是加速机器学习算法学习过程的有效方法之一。它通过使用来自源任务的策略的先验知识来加速学习过程。迁移学习的两个基本问题是如何选择有效的源策略和如何在没有负迁移的情况下重用源策略。在本文中，我们主要讨论后者。在此基础上，提出了一种策略重用方法.在actor-critic中，策略由两个参数集构成：动作偏好和状态值。为了避免负迁移，代理只重用可靠的动作偏好和状态值，暗示首选动作。我们进行简单的实验，以显示所提出的方法的有效性。
—This paper aims to accelerate learning processes of actor-critic method, which is one of major reinforcement learning algorithms, by a transfer learning. Reinforcement learning allows agents to work out the target tasks, autonomously. Transfer learning is one of effective methods to accelerate learning processes of machine learning algorithms. It accelerates learning processes by using prior knowledge from a policy for a source task. Two basic issues for the transfer learning are method to select effective source policies and method to reuse without negative transfer. In this paper, we mainly discuss the latter. We propose the reuse the policy method based on our proposed selection method. In actor-critic, a policy is constructed by two parameter sets: action preferences and state values. To avoid negative transfer, agents reuse only reliable action preferences and state values that imply preferred actions. We perform simple experiments to show the effectiveness of the proposed method.