Learning to Move: Skill Design for Animation and Robotics

有技巧和有目的地移动我们身体的能力很容易被认为是理所当然的。然而，理解和复制这种能力是一个悬而未决的问题。 在计算机图形学和模拟世界中，人体运动往往是虚拟世界中最不真实的方面。 在真实的世界中，与人类和动物相比，机器人的感觉运动能力仍然非常贫乏。 最近，使用深度强化学习（DRL）已经取得了重大进展，尽管与结果相关的重要警告，即，通常仅模拟，视觉质量低，学习效率低，并且非常严格限制运动任务。需要什么方法和进展来赋予物理模拟角色和物理机器人与运动模式和运动技能，如果不是更好的话，可以与人类和动物相比？ 什么是有效创作和设计新技能的正确语言和工具？ 我的研究计划的目标是回答这些问题，在适当的表示，学习算法和数据的帮助下。 用于移动的结构化RL： 一个关键的开放问题是找到或学习好的积木，支持有效学习运动相关的任务。 我将开发和评估的方法：将身体结构和运动学的知识嵌入到政策网络的结构中;利用学到的抽象;探索支持顺序和并行（在时间上）组成的不同政策结构中的学习;并学会将一个新的动力系统与我们已经知道如何控制的规范动力系统相对应。 灵活的自回归模型： 人类运动捕捉数据可用于学习预测未来运动的模型。使用大量的运动数据集合，我们可以学习在任何给定时间点可用的所有运动可能性的模型。使用一类变分自动编码器模型，我们可以使用强化学习（一种试错学习）来学习如何自动生成实现给定任务的逼真运动序列。 模块化模拟到真实的测试平台： 如何使用控制器的新学习能力来考虑以前由于控制能力有限而不切实际的硬件设计？当数据来自一组独立的机器人时，如何实现有效的学习？我们的目标是推进机器人-控制器协同设计的最新技术水平，并测试“模拟到真实的”泛化，即， 将模拟结果转移到真实的机器人上。我们将利用我的研究小组在深度强化学习和模拟到真实的方面的专业知识，并将其扩展到软机器人和模块化机器人。我们将与模块化机器人设计和计算制造方面的专家合作，以实现硬件组件。

The ability to move our bodies with skill and purpose is easily taken for granted as humans. However, understanding and replicating such capabilities is an open problem. In computer graphics and simulated worlds, human motions are often the least-realistic aspect of the virtual worlds being portrayed. In the real world, the sensorimotor capabilities of robots remain greatly impoverished as compared to humans and animals. Recently, significant strides forward have been made using deep reinforcement learning (DRL), albeit with significant caveats related to the results, i.e., often simulation-only, low visual quality, inefficient learning, and very tightly circumscribed motion tasks. What methods and advances are needed to endow physically-simulated characters and physical robots with the movement patterns and motor skills that are comparable to, if not better than, those seen in humans and animals? What is the right languages and tools for effective authoring and design of new skills? The objective of my research program is to answer these questions, with the help of suitable representations, learning algorithms, and data. Structured RL for Movement: A key open problem is that of finding or learning good building blocks that support efficient learning of movement-related tasks. I will develop and evaluate methods that: embed knowlege of body structure and kinematics into the structure of policy networks; leverage learned abstractions; explore learning in different policy structures that support sequential and parallel (in time) composition; and learn to put a new dynamical system into correspondence with a canonical dynamical system that we already know how to control. Flexible Autoregressive Models: Human motion capture data can be used to learn models that are predictive of future movement. Using large collections of motion data, we can learn a model of all motion possibilities that are available at any given point in time. Using a class of variational auto-encoder model, we can then use reinforcement learning, a type of trial-and-error learning, to learn how to automatically produce realistic sequences of movement that achieve given tasks. Modular Sim-to-Real Testbed: How can new learning capabilities for controllers be used to consider hardware designs that previously would have been impractical because of limited control capabilities? How can efficient learning be achieved when the data comes from a set of individually-unique robots? We aim to advance the state of the art in robot-controller co-design, as well as to test for "sim-to-real" generalization, i.e., the transfer of results from simulation onto real robots. We will leverage the expertise of my research group in deep reinforcement learning and sim-to-real, and extend this towards soft robotics and modular robotics. We will collaborate with experts in modular robot design and computational fabrication in order to realize the hardware components.