Learning to Move: Skill Design for Animation and Robotics

作为人类，有技巧和目的地移动身体的能力很容易被认为是理所当然的。然而，理解和复制这些功能是一个有待解决的问题。在计算机图形学和模拟世界中，人类动作通常是所描绘的虚拟世界中最不真实的方面。在现实世界中，与人类和动物相比，机器人的感觉运动能力仍然很差。最近，使用深度强化学习（DRL）取得了重大进展，尽管与结果相关的重大警告，即通常仅用于模拟，视觉质量低，学习效率低，以及非常严格地限制运动任务。我们需要什么样的方法和进步来赋予物理模拟角色和物理机器人与人类和动物的运动模式和运动技能相媲美，如果不是更好的话？什么是正确的语言和工具，可以有效地创作和设计新技能？我的研究计划的目标是在合适的表示、学习算法和数据的帮助下回答这些问题。

The ability to move our bodies with skill and purpose is easily taken for granted as humans. However, understanding and replicating such capabilities is an open problem. In computer graphics and simulated worlds, human motions are often the least-realistic aspect of the virtual worlds being portrayed. In the real world, the sensorimotor capabilities of robots remain greatly impoverished as compared to humans and animals. Recently, significant strides forward have been made using deep reinforcement learning (DRL), albeit with significant caveats related to the results, i.e., often simulation-only, low visual quality, inefficient learning, and very tightly circumscribed motion tasks. What methods and advances are needed to endow physically-simulated characters and physical robots with the movement patterns and motor skills that are comparable to, if not better than, those seen in humans and animals? What is the right languages and tools for effective authoring and design of new skills? The objective of my research program is to answer these questions, with the help of suitable representations, learning algorithms, and data. Structured RL for Movement: A key open problem is that of finding or learning good building blocks that support efficient learning of movement-related tasks. I will develop and evaluate methods that: embed knowlege of body structure and kinematics into the structure of policy networks; leverage learned abstractions; explore learning in different policy structures that support sequential and parallel (in time) composition; and learn to put a new dynamical system into correspondence with a canonical dynamical system that we already know how to control. Flexible Autoregressive Models: Human motion capture data can be used to learn models that are predictive of future movement. Using large collections of motion data, we can learn a model of all motion possibilities that are available at any given point in time. Using a class of variational auto-encoder model, we can then use reinforcement learning, a type of trial-and-error learning, to learn how to automatically produce realistic sequences of movement that achieve given tasks. Modular Sim-to-Real Testbed: How can new learning capabilities for controllers be used to consider hardware designs that previously would have been impractical because of limited control capabilities? How can efficient learning be achieved when the data comes from a set of individually-unique robots? We aim to advance the state of the art in robot-controller co-design, as well as to test for "sim-to-real" generalization, i.e., the transfer of results from simulation onto real robots. We will leverage the expertise of my research group in deep reinforcement learning and sim-to-real, and extend this towards soft robotics and modular robotics. We will collaborate with experts in modular robot design and computational fabrication in order to realize the hardware components.