Learning task-relevant parameters for human and robot motion primitives to acquire complex manipulation skills on a humanoid robot

虽然目前机器人比人类更快、更强壮、更精确，但它们在操纵物体时仍远未达到人类的表现。主要原因是，作为运动控制器的大脑在灵活性、自主学习能力和可靠性方面远远优于机器人控制器。从长远来看，这些特性也是对机器人的必要要求；例如，在老年护理或物理治疗中需要，但也用于设计有效的机器人假肢。一个重要的策略，以实现更高的学习能力和更大的灵活性，以产生一个大的动作曲目集中在运动原语的想法。运动原语是简短的、目标导向的、特定于任务的、复杂性较低的运动。它们可以被排序或叠加，以实现更复杂的运动技能。遗憾的是，机器人运动原语的参数空间往往与任务的成功与否没有直接关系，即参数处于高维抽象空间中，与实际任务目标的关系复杂。因此，本项目的第一个目标是使用降维技术学习运动原语的任务相关参数。这些参数的例子是“在什么高度抓杯子”或“在什么位置抓水龙头”。我们专注于学习与任务相关的人形机器人对象操作。鉴于目前的人形机器人与人类之间的相似性，我们打算用从人类数据中提取的任务相关参数空间来启动机器人探索。低维抽象运动基元参数简化了机器人的运动规划。因此，该项目的第二个主要目标是使机器人能够通过强化学习方法优化整个运动原语序列，特别是来自策略梯度和概率强化学习的想法。优化的运动原语序列将为机器人提供更完整、更精细的复杂运动技能库。

Although robots are currently faster, stronger, and more accurate than humans, they are still far from achieving human-like performance when manipulating objects. The main reason is that the brain as a motion controller is far superior over robotic controllers in terms of flexibility, autonomous learning abilities, and reliability. In the long run, such characteristics are necessary requirements for robots too; as, for instance, needed in elderly care or physical therapy, but also for the design of effective robotic prostheses. An important strategy in achieving higher learning abilities and more flexibility in generating a large movement repertoire has focused on the idea of motion primitives. Motion primitives are short, goal-directed and task-specific movements of reduced complexity. They can be sequenced or superimposed in order to achieve more complex movement skills. Unfortunately, parameter spaces for robot motion primitives are often not directly related to task success, i.e., the parameters are in a high-dimensional abstract space with a complex relationships to actual task goal. Therefore, the first aim of this project is to learn task-relevant parameters for motion primitives using dimensionality reduction techniques. Examples of such parameters are “at which height to grasp a glass” or “at which position to reach for a faucet”. We focus on learning task-relevant object manipulation for humanoid robots. Given the similarity between current humanoid robots and humans, we intend to prime robot exploration with the task-relevant parameter space extracted from human data. Having low-dimensional abstract motion primitives parameters simplifies motion planning for the robot. The second main goal of the project is therefore to enable the robot to optimize whole sequences of motion primitives with reinforcement learning methods, particularly ideas from policy gradients and probabilistic reinforcement learning. Optimized motion primitive sequences will provide robots with a more complete and refined library of complex motor skills.