Flow-based Intrinsic Curiosity Module
Flow-based Intrinsic Curiosity Module
复制标题
基于流程的内在好奇心模块
DOI：
10.24963/ijcai.2020/286
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Chun
中科院分区：
文献类型：
--
作者：
Hsuan;Po;Min;Chun
在本文中，我们重点关注深度强化学习（DRL）框架上基于预测的新颖性估计策略，并提出基于流的内在好奇心模块（FICM），以利用光流估计的预测误差作为探索奖励。我们提出了利用连续观察之间捕获的运动特征来评估环境中观察的新颖性的概念。 FICM 鼓励 DRL 智能体探索具有不熟悉的运动特征的观察，并且在估计新颖性时只需要两个连续帧即可获得足够的信息。我们评估我们的方法，并将其与多个基准环境上的许多现有方法进行比较，包括 Atari 游戏、超级马里奥兄弟和 ViZDoom。我们证明 FICM 有利于以移动物体为特征的任务或环境，这使得 FICM 能够利用连续观察之间的运动特征。我们进一步分析了FICM的编码效率，并全面讨论了其适用领域。请参阅此处查看我们的代码和演示视频。
In this paper, we focus on a prediction-based novelty estimation strategy upon the deep reinforcement learning (DRL) framework, and present a flow-based intrinsic curiosity module (FICM) to exploit the prediction errors from optical flow estimation as exploration bonuses. We propose the concept of leveraging motion features captured between consecutive observations to evaluate the novelty of observations in an environment. FICM encourages a DRL agent to explore observations with unfamiliar motion features, and requires only two consecutive frames to obtain sufficient information when estimating the novelty. We evaluate our method and compare it with a number of existing methods on multiple benchmark environments, including Atari games, Super Mario Bros., and ViZDoom. We demonstrate that FICM is favorable to tasks or environments featuring moving objects, which allow FICM to utilize the motion features between consecutive observations. We further ablatively analyze the encoding efficiency of FICM, and discuss its applicable domains comprehensively. See here for our codes and demo videos.