DEXMAN: Improving robot’s DEXterous MANipulability by learning stiffness-based human motor skills and visuo-tactile exploration

机器人专家已经付出了巨大的努力来模仿人类的手，不仅从形状上，而且从功能上。然而，对未知物体/工具的鲁棒抓取和操纵仍然是一个有待彻底解决的开放性问题。在这个项目中，我们将研究一种基于人类运动技能提取的方法，以实现机器人手臂/手系统的鲁棒灵巧抓取和手操作。基于交互式操作和多模态反馈的基于多传感器融合的自适应抓取和操作控制框架，在与大小不同的物体/工具交互的同时，代表人类的运动技能。形状和刚度，我们将创建一个增强的分层基元库，关于人的手/手臂的刚度，运动和手指步态。基于多模态传感器融合的分层自适应抓取和操作控制方法，通过在线感知反馈，为从人到机械手臂系统的可泛化技能传递提供知识基础。针对不同的物体/工具尺寸、形状和刚度，将实现技能的泛化。物体属性将通过基于视觉/触觉的探索控制方法在线建模。多模态感知反馈也将用作原始库的输入，以推广运动，刚度和步态轨迹。我们将演示提出的抓取和操作方法与典型的日常生活任务，如抓刀和切水果。在人类运动技能学习（SCUT）、基于视觉-触觉的识别与交互（UNIBI）和基于视觉-触觉的多指机械手自适应抓取与灵巧操作（DLR）方面有明确记录的三个研究机构将密切合作，实现这一目标。

Roboticists have made huge efforts to mimic the human hand, not only from the form but also from the functionalities. However, robustly grasping and manipulating an unknown object/tool is still an open question to be thoroughly solved. In this project, we will investigate a human motor skill extraction based approach to achieve robust dexterous grasping and in-hand manipulation on a robotic arm/hand system:  A novel framework of augmented dynamic movement primitives DMPs embedding perception information for human skill extraction and generalization to new tasks  Reconstructing and tracking an unknown object by exploiting interactive manipulation and multi-modal feedback  Multiple sensor fusion based adaptive grasping and manipulation control framework enhanced by human motor skills extractionTo represent human motor skills while interacting with objects/tools that differ in size, shape and stiffness, we will create an augmented hierarchical primitive-based library, with respect to human hand/arm stiffness, motion and finger gaiting. With online perception feedback, this primitives library will provide the knowledge basis for generalizable skills transferring from human to a robotic hand-arm system, by a hierarchical adaptive grasping and manipulation control method based on multi-modal sensor fusion. The skills generalization will be achieved regarding different object/tool sizes, shapes and stiffness. Object properties will be modelled online through visuo/tactile based exploration control method. Multi-modal perception feedback will also be used as input of the primitive library to generalize motion, stiffness, and gaiting trajectories. We will demonstrate the proposed grasping and manipulation approach with a typical daily-of-live task such as grasping a knife and cutting a fruit. Three institutes with a clear record in human motor skills learning (SCUT), visuo-tactile based recognition and interaction (UNIBI), and visuo-tactile based adaptive grasping and dexterous manipulation with multi-fingered robotic hands (DLR), will tightly cooperate towards this aim.