Learning task-relevant parameters for human and robot motion primitives to acquire complex manipulation skills on a humanoid robot

尽管目前机器人比人类更快、更强壮、更准确，但它们在操纵物体时仍远未达到人类的表现。主要原因是，作为运动控制器的大脑在灵活性、自主学习能力和可靠性方面远远优于机器人控制器。从长远来看，这些特征也是对机器人的必要要求；例如，在老年护理或物理治疗中需要这样的特征，但也是设计有效的机器人假肢所必需的。在生成大型动作曲目时实现更高的学习能力和更大的灵活性的一个重要策略是关注动作基元的想法。运动基元是简短的、以目标为导向的、特定于任务的运动，复杂性较低。它们可以被排序或叠加，以实现更复杂的运动技能。遗憾的是，机器人运动基元的参数空间往往与任务成功与否没有直接关系，即参数处于高维抽象空间，与实际任务目标有着复杂的关系。因此，本项目的第一个目标是使用降维技术学习运动基元的与任务相关的参数。这类参数的例子有“握住玻璃杯的高度”或“伸手拿水龙头的位置”。我们的重点是学习类人机器人与任务相关的对象操作。考虑到目前的类人机器人与人类之间的相似性，我们打算用从人类数据中提取的与任务相关的参数空间来启动机器人的探索。具有低维抽象运动基元参数简化了机器人的运动规划。因此，该项目的第二个主要目标是使机器人能够使用强化学习方法来优化整个运动基元序列，特别是来自策略梯度和概率强化学习的想法。优化的运动基元序列将为机器人提供更完整和更精细的复杂运动技能库。

Although robots are currently faster, stronger, and more accurate than humans, they are still far from achieving human-like performance when manipulating objects. The main reason is that the brain as a motion controller is far superior over robotic controllers in terms of flexibility, autonomous learning abilities, and reliability. In the long run, such characteristics are necessary requirements for robots too; as, for instance, needed in elderly care or physical therapy, but also for the design of effective robotic prostheses. An important strategy in achieving higher learning abilities and more flexibility in generating a large movement repertoire has focused on the idea of motion primitives. Motion primitives are short, goal-directed and task-specific movements of reduced complexity. They can be sequenced or superimposed in order to achieve more complex movement skills. Unfortunately, parameter spaces for robot motion primitives are often not directly related to task success, i.e., the parameters are in a high-dimensional abstract space with a complex relationships to actual task goal. Therefore, the first aim of this project is to learn task-relevant parameters for motion primitives using dimensionality reduction techniques. Examples of such parameters are “at which height to grasp a glass” or “at which position to reach for a faucet”. We focus on learning task-relevant object manipulation for humanoid robots. Given the similarity between current humanoid robots and humans, we intend to prime robot exploration with the task-relevant parameter space extracted from human data. Having low-dimensional abstract motion primitives parameters simplifies motion planning for the robot. The second main goal of the project is therefore to enable the robot to optimize whole sequences of motion primitives with reinforcement learning methods, particularly ideas from policy gradients and probabilistic reinforcement learning. Optimized motion primitive sequences will provide robots with a more complete and refined library of complex motor skills.