AIS:Learning Motor Skills from Trajectory-based Reinforcement Learning

这项研究解决了未来复杂的机器人系统，如类人辅助机器人，如何获得、改进和保持各种运动技能，使它们能够在正常的人类环境中自主操作的问题。人类在执行运动技能的能力方面表现出色，这是由于各个方面，包括i)模仿学习，使他们能够将关于任务的先验知识从老师转移到学生，ii)试错学习，为他们提供改进技能的手段，iii)反应性行为，可以处理动态和随机环境，以及iv)顺从控制。这是一种抗干扰的基本机制，并促进了与他人一起行动的安全性。了解这些能力的基本机制将导致真正自主机器人系统的技术进步。我们的技术工作包括研究运动原语方面的运动控制的模块化表示，研究基于轨迹的强化学习的运动原语和运动原语序列的试错改进，使用概率强化学习和路径积分强化学习的新技术，研究使用运动原语与感知变量直接耦合的反应性行为，在操作空间控制器的帮助下进行兼容控制，这是可以学习的。除了传统的基准模拟研究，我们的评估将包括一个全身人形机器人的运动技能学习，这是一个极大地挑战我们方法可扩展性的系统。

This research addresses the question of how complex future robotic systems, e.g., like humanoid assistive robots, can acquire, refine, and maintain a variety of motor skills that enable them to operate autonomously in normal human environments. Humans excel in their abilities to perform motor skills due to various aspects, including i) imitation learning, which allows them to transfer prior knowledge about a task from a teacher to a student, ii) trial-and-error learning, which provides them with means to refine skills, iii) reactive behaviors, which can deal with dynamic and stochastic environments, and iv) compliant control, which is a basic mechanism for robustness against disturbances and promotes safety to act amongst other humans. Understanding the basic mechanisms of these abilities will lead to technological advances towards truly autonomous robotic systems.Our technical work includes research on modular representations of motor control in terms of movement primitives, research on trial-and-error improvement of motor primitives and sequences of motor primitives with trajectory-based reinforcement learning using novel techniques from probabilistic reinforcement learning and path-integral reinforcement learning, research on reactive behavior using direct coupling of motor primitives to perceptual variables, and compliant control with the help of operational space controllers that can be learned. Besides traditional benchmark simulation studies, our evaluations will include the learning of motor skills with a full-body humanoid robot, a system that significantly challenges the scalability of our methods.