Learning task-relevant parameters for human and robot motion primitives to acquire complex manipulation skills on a humanoid robot

尽管机器人目前比人类更快、更强、更准确，但在操纵物体时，它们仍远未达到人类的表现。主要原因是大脑作为运动控制器在灵活性、自主学习能力和可靠性方面远远上级机器人控制器。从长远来看，这些特性也是对机器人的必要要求;例如，老年人护理或物理治疗所需要的，以及有效的机器人假肢的设计。一个重要的策略，在实现更高的学习能力和更大的灵活性，在产生一个大的运动剧目集中在运动基元的想法。运动基元是简短的、目标导向的和特定于任务的运动，其复杂性降低。它们可以被排序或叠加，以实现更复杂的运动技能。不幸的是，机器人运动基元的参数空间通常与任务成功不直接相关，即，这些参数是在一个高维抽象空间中，与实际任务目标有着复杂的关系。因此，本项目的第一个目标是使用降维技术学习与任务相关的运动基元参数。这样的参数的示例是“在哪个高度抓住玻璃杯”或“在哪个位置够到水龙头”。我们专注于学习任务相关的人形机器人的对象操作。鉴于目前的类人机器人和人类之间的相似性，我们打算用从人类数据中提取的任务相关参数空间来启动机器人探索。具有低维抽象运动基元参数简化了机器人的运动规划。因此，该项目的第二个主要目标是使机器人能够使用强化学习方法优化整个运动基元序列，特别是来自策略梯度和概率强化学习的想法。优化的运动基元序列将为机器人提供更完整和更精细的复杂运动技能库。

Although robots are currently faster, stronger, and more accurate than humans, they are still far from achieving human-like performance when manipulating objects. The main reason is that the brain as a motion controller is far superior over robotic controllers in terms of flexibility, autonomous learning abilities, and reliability. In the long run, such characteristics are necessary requirements for robots too; as, for instance, needed in elderly care or physical therapy, but also for the design of effective robotic prostheses. An important strategy in achieving higher learning abilities and more flexibility in generating a large movement repertoire has focused on the idea of motion primitives. Motion primitives are short, goal-directed and task-specific movements of reduced complexity. They can be sequenced or superimposed in order to achieve more complex movement skills. Unfortunately, parameter spaces for robot motion primitives are often not directly related to task success, i.e., the parameters are in a high-dimensional abstract space with a complex relationships to actual task goal. Therefore, the first aim of this project is to learn task-relevant parameters for motion primitives using dimensionality reduction techniques. Examples of such parameters are “at which height to grasp a glass” or “at which position to reach for a faucet”. We focus on learning task-relevant object manipulation for humanoid robots. Given the similarity between current humanoid robots and humans, we intend to prime robot exploration with the task-relevant parameter space extracted from human data. Having low-dimensional abstract motion primitives parameters simplifies motion planning for the robot. The second main goal of the project is therefore to enable the robot to optimize whole sequences of motion primitives with reinforcement learning methods, particularly ideas from policy gradients and probabilistic reinforcement learning. Optimized motion primitive sequences will provide robots with a more complete and refined library of complex motor skills.