Learning to move as a human: one-shot learning of human motion

用于人体运动分析和合成的计算模型在医疗保健、计算机图形学和机器人技术等领域都有应用。在医疗保健中，对人体运动的分析可以用于例如跟踪老年人的运动衰退。在计算机图形学中，人体运动分析可以用于当测量可能由于遮挡而有噪声或丢失时从单个相机跟踪人体姿势。在机器人技术中，人体运动分析和合成可以通过模仿人类的演示来教授机器人新技能，从而减少对工业机器人或服务机器人进行编程所需的工作量。了解人类如何运动的一种方法是收集特定人类活动的示例，并设计一个机器学习模型，从这些示例中提取模式。我们收集的例子越多，模型就越有可能在数据中找到可用于解决预测任务的共同特征。然而，在需要人体运动分析和合成的不同应用中，特别是在通过演示进行机器人编程中，收集许多示例是昂贵且耗时的。也就是说，我们希望机器人学习一项新技能时，尽可能少的演示，更像人类。事实上，人类通过模仿一个或几个例子来有效地学习，这一点通过他们生成新例子或创建以前用于模仿的例子中没有的抽象运动的能力得到了进一步验证。在这个项目中，我们的目标是使用一次性学习的认知科学概念来开发一个数据高效的机器学习模型。在认知科学中，单次学习（OL）是指使用一个或几个示例构建智能代理的想法。使用这个概念来构建数据高效模型的成功例子包括用于生成语音概念和具有类似人类外观的手写字符的OL模型。认知科学的最新研究表明，人类通过结合应用于原始概念的三个核心原则来实现OL：因果关系，组合性和“学会学习”。它还声称，这些成分可以在生产复制人类智能的机器学习模型中发挥积极作用。我们将通过该提案的两个关键创新来实现我们的目标：（i）同时结合因果关系的通用方法，组合性和学习运动基元的学习，以及（ii）使用物理启发的高斯过程（GP）的特定实例化关于（i），尽管存在并入OL的一些成分的机器学习模型，但是尚未提出它们的同时组合以构建用于人类运动分析和合成的数据高效模型。关于（ii），我们的运动原语的GP表示使用具有两个特征的物理学启发的协方差函数：由于其非参数性质而有效地使用数据;以及包含OL的因果关系原理，为轨迹数据提供生成机制。这些GP运动原语的组合性将采用正式语言理论的思想，特别是隐马尔可夫模型与明确的状态持续时间。为了在实践中使用该模型，我们将提供一个统计推断框架，用于将OL模型的参数拟合到给定数据，并用于计算预测的概率分布。我们将测试OL模型的性能，用于与动作捕捉数据相关的不同任务，以及使用拟人机器人的动觉演示进行模仿学习。我们的结果将是完全可复制的，我们的软件将作为开源发布。

Computational models for human motion analysis and synthesis have applications in fields as diverse as healthcare, computer graphics, and robotics. In healthcare, analysis of human movements can be used, for example, for tracking motor decline in the elderly. In computer graphics, human motion analysis can be used for human pose tracking from a single camera when measurements might be noisy or missing due to occlusion. In robotics, human motion analysis and synthesis can be used for teaching robots new skills by imitating demonstrations of a human, reducing the effort required to program an industrial robot or a service robot.One approach to understand how humans move consists of collecting examples of a particular human activity and designing a machine learning model that extracts patterns from those examples. The more examples we collect, the more likely it is for the model to find common features in the data that can be exploited for solving predictive tasks. However, in different applications that require human motion analysis and synthesis, particularly in robot programming by demonstration, collecting many examples is expensive and time-consuming. I.e. we would like a robot to learn a new skill with as few demonstrations as possible, more like a human does. Indeed, humans learn efficiently by imitation with just one or few examples, which is further validated by their ability to generate new examples or creating abstract motions that were not previously seen in the examples that were used to imitate.In this project, our objective is to develop a data-efficient machine learning model for human motion using the cognitive science concept of one-shot learning.In cognitive science, one-shot learning (OL) refers to the idea of building intelligent agents using one or few examples. Successful illustrations of the use of this concept for building data efficient models include OL models for generating speech concepts and handwritten characters with human-like appearance. Recent research in cognitive science suggests that humans achieve OL through the combination of three core principles applied to primitive concepts: causality, compositionality, and "learning to learn". It also claims that these ingredients could play an active role in producing machine learning models that replicate human intelligence.We will achieve our objective through the two key novelties of this proposal: (i) a generic methodology that simultaneously combines causality, compositionality and learning to learn of motor primitives and (ii) a particular instantiation that uses physics-inspired Gaussian process (GP) representations of such motor primitives.With respect to (i), although there are machine learning models that incorporate some of the ingredients of OL, their simultaneous combination to build data-efficient models for human motion analysis and synthesis has not been proposed yet. With respect to (ii), our GP representation of a motor primitive uses a physics-inspired covariance function with two features: the efficient use of data due to its non-parametric nature; and the inclusion of the principle of causality of OL, providing a generative mechanism for trajectory data. Compositionality of these GP motor primitives will be approached using ideas from formal language theory, in particular, hidden Markov models with explicit state durations. Learning to learn will be accomplished by providing hierarchies of such hidden Markov models.In order to use the model in practice, we will provide a statistical inference framework for fitting the parameters of the OL model to given data, and for computing probability distributions for prediction. We will test the performance of the OL model for different tasks related to motion capture data, and for imitation learning using kinesthetic demonstrations from anthropomorphic robots. Our results will be fully reproducible and our software to be released as open source.