Causal Models for Generalizable Robot Learning

该提案旨在构建智能机器来演示因果推理，特别是在新的环境中。人类是非常高效的学习系统！支持我们学习效率的一个论据是因果关系，也就是说，从单纯的观察中推断因果关系的能力。因果模型超越了统计依赖结构的表示，转向支持干预、计划和推理的模型，实现了康拉德·洛伦兹的思维概念，即在想象的空间中行动。这种因果知识的获得得到了强大的学习机制的支持，使我们能够有效地将新的证据与先前的信念结合起来，并将行为推广到新的问题实例。理解和回答关于观测值的潜在机制的问题，或者基于对环境变量子集的操纵来预测观测值的变化，是因果推理的本质。构建这样的因果模型不仅是计算机科学关注的焦点，也是物理和生物科学以及心理学和哲学关注的焦点。然而，这些努力中的一些只集中在观察数据上，并且由于需要人类进行对照试验而在规模上受到限制。 因果模型与传统的统计学习方法不同，传统的统计学习方法研究一组变量P（X，Y）的联合分布，目标是近似量，例如E（Y| X）在一个函数类下。然而，如果这种关联是在忽略潜在生成机制的情况下学习的，那么我们就会做出虚假的决定，比如一个在线系统建议我们买一辆汽车，因为我们买了轮胎，因为它们“经常一起买”。因果学习考虑了更丰富的假设类别，并试图利用联合分布具有对应于结构分配的因果因子分解的事实。这导致了更多的信息，通过干预和反事实分析来执行更好的泛化，这是统计学习模型所不具备的。计算效率高的因果学习是驱动下一个十年交互式机器人学习的核心问题。 - 如何更好地从半监督和自监督数据中学习？- 我们如何将概念和技能政策推广到新领域？ 如何有效地进行持续的多任务学习？ - 我们如何在RL中利用大型离线观测数据集？

This proposal aims to build intelligent machines to demonstrate cause-effect reasoning, particularly in novel environments. Humans are remarkably efficient learning systems! An argument in support of our learning efficiency is causality, that is, the ability to infer causation from mere observations. Causal models go beyond the representation of statistical dependence structures towards models that support intervention, planning, and reasoning, realizing Konrad Lorenz' notion of thinking as acting in an imagined space. The acquisition of this causal knowledge is supported by powerful learning mechanisms that allow us to effectively integrate novel evidence with prior beliefs, and generalize behavior to novel problems instances. Understanding and answering questions about the latent mechanisms by which observations take on values or predicting the change in observations based on manipulation of a subset of environment variables is the essence of causal inference. And constructing such causal models has been the focus of attention not only in computer science, but also in physical and biological sciences, as well as psychology and philosophy. However a number of these efforts have only focussed on observational data, and are limited in scale by the need for humans to perform controlled trials. Causal models are distinct from traditional statistical learning methods that study the joint distribution of a set of variables, P(X,Y) with the objective of approximates quantities such as E(Y|X) under a function class. However if such associations are learned disregarding the the underlying generative mechanism, we are left with spurious decisions such as an online system suggesting that we buy a car because we bought tires, since they are "frequently bought together". Causal learning considers a richer class of assumptions, and seeks to exploit the fact that the joint distribution possesses a causal factorization corresponding to the structural assignments. This results in more information for performing better generalization through intervention and counterfactual analysis which is not available to statistical learning models. Computationally efficient causal learning is a central problem driving the next decade of Interactive Robot Learning. - How to learn better from semi-supervised and self-supervised data? -How do we generalize concepts and skill policies to new domains? -How can we efficiently perform continual multi-task learning? -How can we leverage large offline observational datasets in RL?