AIS:Learning Motor Skills from Trajectory-based Reinforcement Learning

AIS：从基于轨迹的强化学习中学习运动技能

• 批准号: 0926052
• 负责人: Stefan Schaal
• 金额: $ 33万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-15 至 2017-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0926052&HistoricalAwards=false
• 关键词: AIS; Learning; Motor; Skills; Trajectory

This research addresses the question of how complex future robotic systems, e.g., like humanoid assistive robots, can acquire, refine, and maintain a variety of motor skills that enable them to operate autonomously in normal human environments. Humans excel in their abilities to perform motor skills due to various aspects, including i) imitation learning, which allows them to transfer prior knowledge about a task from a teacher to a student, ii) trial-and-error learning, which provides them with means to refine skills, iii) reactive behaviors, which can deal with dynamic and stochastic environments, and iv) compliant control, which is a basic mechanism for robustness against disturbances and promotes safety to act amongst other humans. Understanding the basic mechanisms of these abilities will lead to technological advances towards truly autonomous robotic systems.Our technical work includes research on modular representations of motor control in terms of movement primitives, research on trial-and-error improvement of motor primitives and sequences of motor primitives with trajectory-based reinforcement learning using novel techniques from probabilistic reinforcement learning and path-integral reinforcement learning, research on reactive behavior using direct coupling of motor primitives to perceptual variables, and compliant control with the help of operational space controllers that can be learned. Besides traditional benchmark simulation studies, our evaluations will include the learning of motor skills with a full-body humanoid robot, a system that significantly challenges the scalability of our methods.

这项研究解决的问题是，未来复杂的机器人系统，例如人形辅助机器人，可以获得、改进和保持各种运动技能，使它们能够在正常的人类环境中自主操作。人类在执行运动技能方面的能力出类拔萃，原因包括：i)模仿学习，允许他们将关于任务的先验知识从老师传递给学生；ii)试错学习，为他们提供改进技能的手段；iii)反应行为，可以处理动态和随机环境；以及iv)顺从控制，这是抵抗干扰的基本机制，并促进与其他人一起行动的安全性。我们的技术工作包括基于运动基元的运动控制模块表示的研究，使用概率强化学习和路径积分强化学习的基于轨迹的强化学习来改进运动基元和运动基元序列的研究，利用运动基元与感知变量的直接耦合来研究反应行为，以及借助可学习的操作空间控制器进行顺应控制的研究。除了传统的基准模拟研究，我们的评估将包括使用全身类人机器人学习运动技能，这一系统对我们方法的可扩展性提出了重大挑战。