Causal Models for Generalizable Robot Learning

这项提议旨在建造智能机器来演示因果推理，特别是在新的环境中。人类是非常高效的学习系统！支持我们学习效率的一个论点是因果关系，即仅从观察中推断因果关系的能力。因果模型超越了统计依赖结构的表示，而是支持干预、计划和推理的模型，实现了康拉德·洛伦茨的思维是在想象的空间中行动的概念。这种因果知识的获得得到了强大的学习机制的支持，这些机制允许我们有效地将新的证据与先前的信念相结合，并将行为概括为新的问题实例。理解和回答关于观察值的潜在机制的问题，或基于对环境变量子集的操纵来预测观察值的变化，是因果推理的本质。而构建这样的因果模型不仅是计算机科学关注的焦点，也是物理和生物科学以及心理学和哲学关注的焦点。然而，这些努力中的一些只集中在观测数据上，而且由于需要人类进行对照试验，规模有限。因果模型不同于传统的统计学习方法，传统的统计学习方法研究一组变量P(X，Y)的联合分布，目标是在函数类下近似E(Y|X)等量。然而，如果不考虑潜在的生成机制来学习这种联想，我们就会做出错误的决定，比如一个在线系统建议我们买车是因为我们买了轮胎，因为轮胎是“经常一起买的”。因果学习考虑了一类更丰富的假设，并试图利用这样一个事实，即联合分布具有与结构分配相对应的因果分解。这产生了更多的信息，以便通过干预和反事实分析更好地进行概括，这是统计学习模型所不具备的。计算效率高的因果学习是推动下一个十年交互机器人学习的核心问题。-如何从半监督和自我监督的数据中更好地学习？-我们如何将概念和技能政策推广到新的领域？-我们如何高效地进行持续的多任务学习？-我们如何在RL中利用大型离线观测数据集？

This proposal aims to build intelligent machines to demonstrate cause-effect reasoning, particularly in novel environments. Humans are remarkably efficient learning systems! An argument in support of our learning efficiency is causality, that is, the ability to infer causation from mere observations. Causal models go beyond the representation of statistical dependence structures towards models that support intervention, planning, and reasoning, realizing Konrad Lorenz' notion of thinking as acting in an imagined space. The acquisition of this causal knowledge is supported by powerful learning mechanisms that allow us to effectively integrate novel evidence with prior beliefs, and generalize behavior to novel problems instances. Understanding and answering questions about the latent mechanisms by which observations take on values or predicting the change in observations based on manipulation of a subset of environment variables is the essence of causal inference. And constructing such causal models has been the focus of attention not only in computer science, but also in physical and biological sciences, as well as psychology and philosophy. However a number of these efforts have only focussed on observational data, and are limited in scale by the need for humans to perform controlled trials. Causal models are distinct from traditional statistical learning methods that study the joint distribution of a set of variables, P(X,Y) with the objective of approximates quantities such as E(Y|X) under a function class. However if such associations are learned disregarding the the underlying generative mechanism, we are left with spurious decisions such as an online system suggesting that we buy a car because we bought tires, since they are "frequently bought together". Causal learning considers a richer class of assumptions, and seeks to exploit the fact that the joint distribution possesses a causal factorization corresponding to the structural assignments. This results in more information for performing better generalization through intervention and counterfactual analysis which is not available to statistical learning models. Computationally efficient causal learning is a central problem driving the next decade of Interactive Robot Learning. - How to learn better from semi-supervised and self-supervised data? -How do we generalize concepts and skill policies to new domains? -How can we efficiently perform continual multi-task learning? -How can we leverage large offline observational datasets in RL?