Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning
Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning
复制标题
DOI：
10.1609/aaai.v34i02.5483
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
Mark Edmonds;Xiaojian Ma;Siyuan Qi;Yixin Zhu;Hongjing Lu;Song-Chun Zhu
中科院分区：
文献类型：
--
作者：
Mark Edmonds;Xiaojian Ma;Siyuan Qi;Yixin Zhu;Hongjing Lu;Song-Chun Zhu
在相似但不同的环境中学习可转移的知识是广义智能的基本组成部分。在本文中，我们从因果理论的角度来探讨迁移学习的挑战。我们的智能体被赋予了两个基本但通用的迁移学习理论：（i）任务共享一个通用的抽象结构，该结构在各个领域都是不变的，（ii）环境的特定特征的行为在各个领域都保持不变。我们采用贝叶斯因果理论归纳的观点，并使用这些理论在环境之间转移知识。给定这些一般理论，目标是通过交互式地探索问题空间来训练代理，以（i）发现，形成和转移有用的抽象和结构化知识，以及（ii）从环境中观察到的实例级属性中诱导有用的知识。贝叶斯结构的层次结构是用来模拟抽象层次的结构因果知识，和一个实例级的联想学习计划学习哪些特定的对象可以用来诱导状态变化，通过相互作用。这个模型学习计划，然后集成了一个基于模型的计划，以实现在OpenLock环境中的任务，一个虚拟的“逃生室”与复杂的层次结构，需要代理的原因抽象的，广义的因果结构。我们将性能与一组主流的无模型强化学习（RL）算法进行比较。RL代理在不同的试验中表现出较差的转移学习知识的能力。然而，所提出的模型显示了与人类学习者相似的表现趋势，更重要的是，证明了跨试验和学习情况的迁移行为。
Learning transferable knowledge across similar but different settings is a fundamental component of generalized intelligence. In this paper, we approach the transfer learning challenge from a causal theory perspective. Our agent is endowed with two basic yet general theories for transfer learning: (i) a task shares a common abstract structure that is invariant across domains, and (ii) the behavior of specific features of the environment remain constant across domains. We adopt a Bayesian perspective of causal theory induction and use these theories to transfer knowledge between environments. Given these general theories, the goal is to train an agent by interactively exploring the problem space to (i) discover, form, and transfer useful abstract and structural knowledge, and (ii) induce useful knowledge from the instance-level attributes observed in the environment. A hierarchy of Bayesian structures is used to model abstract-level structural causal knowledge, and an instance-level associative learning scheme learns which specific objects can be used to induce state changes through interaction. This model-learning scheme is then integrated with a model-based planner to achieve a task in the OpenLock environment, a virtual “escape room” with a complex hierarchy that requires agents to reason about an abstract, generalized causal structure. We compare performances against a set of predominate model-free reinforcement learning (RL) algorithms. RL agents showed poor ability transferring learned knowledge across different trials. Whereas the proposed model revealed similar performance trends as human learners, and more importantly, demonstrated transfer behavior across trials and learning situations.1