Robot In-hand Dexterous manipulation by extracting data from human manipulation of objects to improve robotic autonomy and dexterity - InDex

机器人手动灵巧操纵，通过从人类操纵物体中提取数据来提高机器人的自主性和灵活性 - InDex

• 批准号: EP/S032355/1
• 负责人: Diego Resende Faria
• 金额: $ 45.71万
• 依托单位: Aston University
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2019
• 资助国家: 英国
• 起止时间: 2019 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FS032355%2F1
• 关键词: Robot; hand; Dexterous; manipulation; extracting

Humans excel when dealing with everyday objects and manipulation tasks, learning new skills, and adapting to different or complex environments. This is a basic skill for our survival as well as a key feature in our world of artefacts and human-made devices. Our expert ability to use our hands results from a lifetime of learning by both observing other skilled humans and ourselves as we discover how to handle objects first hand. Unfortunately, today's robotic hands are still unable to achieve such a high level of dexterity in comparison to humans nor are systems entirely able to understand their own potential. In order for robots to truly operate in a human world and fulfil the expectations as intelligent assistants, they must be able to manipulate a wide variety of unknown objects by mastering their capabilities of strength, finesse and subtlety. To achieve such dexterity with robotic hands, cognitive capacity is needed to deal with uncertainties in the real world and to generalise previously learned skills to new objects and tasks. Furthermore, we assert that the complexity of programming must be greatly reduced and robot autonomy must become much more natural. The InDex project aims to understand how humans perform in-hand object manipulation and to replicate the observed skilled movements with dexterous artificial hands, merging the concepts of deep reinforcement and transfer learning to generalise in-hand skills for multiple objects and tasks. In addition, an abstraction and representation of previous knowledge will be fundamental for the reproducibility of learned skills to different hardware. Learning will use data across multiple modalities that will be collected, annotated and assembled into a large dataset. The data and our methods will be shared with the wider research community to allow testing against benchmarks and reproduction of results. More concretely, the core objectives are: (i) to build a multi-modal artificial perception architecture that extracts data of object manipulation by humans; (ii) the creation of a multimodal dataset of in-hand manipulation tasks such as regrasping, reorienting and finely repositioning; (iii) the development of an advanced object modelling and recognition system, including the characterisation of object affordances and grasping properties, in order to encapsulate both explicit information and possible implicit object usages; (iv) to autonomously learn and precisely imitate human strategies in handling tasks; and (v) to build a bridge between observation and execution, allowing deployment that is independent of the robot architecture.

人类擅长处理日常物品和操作任务，学习新技能，适应不同或复杂的环境。这是我们生存的基本技能，也是我们这个人工制品和人造设备世界的一个关键特征。我们使用双手的专业能力来自于一生的学习，通过观察其他熟练的人类和我们自己，我们发现了如何直接处理物体。不幸的是，与人类相比，今天的机器人手仍然无法实现如此高的灵活性，系统也无法完全理解自己的潜力。为了让机器人真正在人类世界中运作，并实现作为智能助手的期望，它们必须能够通过掌握力量，技巧和微妙的能力来操纵各种各样的未知物体。要用机器人手实现这种灵巧，需要认知能力来处理真实的世界中的不确定性，并将以前学到的技能推广到新的对象和任务。此外，我们断言，编程的复杂性必须大大降低，机器人的自主性必须变得更加自然。InDex项目旨在了解人类如何执行手内物体操作，并使用灵巧的人造手复制观察到的熟练动作，融合深度强化和迁移学习的概念，以推广多个物体和任务的手内技能。此外，先前知识的抽象和表示将是学习技能在不同硬件上再现的基础。学习将使用跨多种模式的数据，这些数据将被收集，注释和组装成一个大型数据集。我们的数据和方法将与更广泛的研究界共享，以便对基准进行测试并复制结果。更具体地说，核心目标是：（i）建立一个多模态人工感知体系结构，提取人类操纵对象的数据;（ii）创建一个多模态数据集的手操纵任务，如重握，重定向和精细重新定位;（iii）开发先进的物体建模和识别系统，包括物体可供性和抓握特性的表征，以便封装显式信息和可能的隐式对象使用;（iv）自主学习并精确模仿处理任务的人类策略;以及（v）在观察和执行之间建立桥梁，允许独立于机器人架构的部署。

项目成果

Computer Vision Systems - 12th International Conference, ICVS 2019, Thessaloniki, Greece, September 23-25, 2019, Proceedings

计算机视觉系统 - 第十二届国际会议，ICVS 2019，希腊塞萨洛尼基，2019 年 9 月 23-25 日，会议记录

• DOI: 10.1007/978-3-030-34995-0_35
• 发表时间: 2019
• 作者: Bauer D

SyDPose: Object Detection and Pose Estimation in Cluttered Real-World Depth Images Trained using Only Synthetic Data

SyDPose：仅使用合成数据训练的杂乱的现实世界深度图像中的对象检测和姿势估计

• DOI: 10.1109/3dv.2019.00021
• 发表时间: 2019
• 作者: Thalhammer S

Advances in Information and Communication - Proceedings of the 2021 Future of Information and Communication Conference (FICC), Volume 2

信息和通信的进展 - 2021 年信息和通信未来会议 (FICC) 论文集，第 2 卷

• DOI: 10.1007/978-3-030-73103-8_65
• 发表时间: 2021
• 作者: Dolopikos C

Goal Density-based Hindsight Experience Prioritization for Multi-Goal Robot Manipulation Reinforcement Learning

基于目标密度的后见之明多目标机器人操作强化学习经验优先级

• 发表时间: 2020
• 作者: Kuang I.

Dynamic Fusion of Electromyographic and Electroencephalographic Data towards Use in Robotic Prosthesis Control

• DOI: 10.1088/1742-6596/1828/1/012056
• 发表时间: 2021-02
• 期刊: Journal of Physics: Conference Series
• 作者: Michael Pritchard;Abraham Itzhak Weinberg;John A R Williams;F. Campelo;Harry Goldingay;D. Faria