Structural knowledge transfer by spatial abstraction for reinforcement learning agents
Structural knowledge transfer by spatial abstraction for reinforcement learning agents
复制标题
强化学习代理的空间抽象结构知识转移
作者：
L. Frommberger;D. Wolter
在本文中，我们研究了抽象原则在智能体控制学习任务中的作用。我们从形式化的角度分析抽象，并描述了三个不同的方面：特殊化、粗化和概念分类。我们开发的分类法使我们能够将现有的抽象方法相互关联，从而产生设计支持知识转移的知识表示的实践代码。我们详细介绍了如何利用特殊化来实现强化学习中的知识转移。我们建议使用所谓的结构空间方面化知识表示来解释状态空间的结构属性，并提出一种后验结构空间方面化（APSST）作为一种从学习策略中提取一般明智行为的方法。这个新策略可以用于知识转移，以支持在不同环境中学习新任务。最后，我们提出了一个案例研究，展示了从简单的模拟到现实世界机器人平台的一般明智的导航技能的转移。
In this article we investigate the role of abstraction principles for knowledge transfer in agent control learning tasks. We analyze abstraction from a formal point of view and characterize three distinct facets: aspectualization, coarsening, and conceptual classification. The taxonomy we develop allows us to interrelate existing approaches to abstraction, leading to a code of practice for designing knowledge representations that support knowledge transfer. We detail how aspectualization can be utilized to achieve knowledge transfer in reinforcement learning. We propose the use of so-called structure space aspectualizable knowledge representations that explicate structural properties of the state space and present a posteriori structure space aspectualization (APSST) as a method to extract generally sensible behavior from a learned policy. This new policy can be used for knowledge transfer to support learning new tasks in different environments. Finally, we present a case study that demonstrates transfer of generally sensible navigation skills from simple simulation to a real-world robotic platform.