TRANSFER LEARNING BASED ON FORBIDDEN RULE SET IN ACTOR-CRITIC METHOD
TRANSFER LEARNING BASED ON FORBIDDEN RULE SET IN ACTOR-CRITIC METHOD
复制标题
基于行为批评法禁止规则集的迁移学习
DOI：
--
复制
发表时间：
2011
期刊：
影响因子：
--
通讯作者：
S. Tsuruoka
中科院分区：
文献类型：
--
作者：
Toshiaki Takano;H. Takase;T. Hayashi;S. Tsuruoka
在本文中，我们的目标是加快行动者-批评者方法的学习过程。我们提出了有效的迁移学习方法，它通过使用从源任务中获取的信息来减少训练周期。该方法包括两个思想，选择一个政策转移的方法，并考虑到每个演员批评参数集的特点转移方法。该选择方法旨在减少在选择阶段和训练阶段中使用的冗余试验和错误。我们引入了禁止的规则集，这是很容易检测到的训练阶段，和一致率，衡量一个源政策的有效性。转移方法的目的是将选定的源策略合并到目标策略，而不进行负转移。它只传输可靠的操作首选项和暗示首选操作的状态值。我们通过简单的实验证明了该方法的有效性。代理人从数据库中找到有效的策略，并完成了他们的训练，与原来的演员-评论家的方法相比，更少或相同的情节。
In this paper, we aim to accelerate learning processes in actor-critic method. We proposed the effective transfer learning method, which reduces training cycles by using information acquired from source tasks. The proposed method consists of two ideas, the method to select a policy to transfer, and the transfer method considering the characteristic of each actor-critic parameter set. The selection method aims to reduce redundant trial and error that are used in the selection phase and the training phase. We introduce the forbidden rule set, which are detected easily in the training phase, and concordance rate that measures an effectiveness of a source policy. The transfer method aims to merge a selected source policy to the target policy without negative transfers. It transfers only reliable action preferences and state values that implies preferred actions. We show the effectiveness of the proposed method by simple experiments. Agents found effective policies from the database, and finished their training with less or same episodes than the original actor-critic method.