Collaborative Research: NRI: FND: Learning Graph Neural Networks for Multi-Object Manipulation

合作研究：NRI：FND：学习多对象操作的图神经网络

基本信息

批准号：
2024778
负责人：
Tucker Hermans
金额：
$ 34.43万
依托单位：
University of Utah
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2024778&HistoricalAwards=false
关键词：
Collaborative Research NRI FND Learning

项目摘要

For robots to act as ubiquitous assistants in daily life, they must regularly contend with environments involving many objects and objects built of many constituent parts. Current robotics research focuses on providing solutions to isolated manipulation tasks, developing specialized representations that do not readily work across tasks. This project seeks to enable robots to learn to represent and understand the world from multiple sensors, across many manipulation tasks. Specifically, the project will examine tasks in heavily cluttered environments that require multiple distinct picking and placing actions. This project will develop autonomous manipulation methods suitable for use in robotic assistants. Assistive robots stand to make a substantial impact in increasing the quality of life of older adults and persons with certain degenerative diseases. These methods also apply to manipulation in natural or man-made disasters areas, where explicit object models are not available. The tools developed in this project can also improve robot perception, grasping, and multi-step manipulation skills for manufacturing.With their ability to learn powerful representations from raw perceptual data, deep neural networks provide the most promising framework to approach key perceptual and reasoning challenges underlying autonomous robot manipulation. Despite their success, existing approaches scale poorly to the diverse set of scenarios autonomous robots will handle in natural environments. These current limitations of neural networks arise from being trained on isolated tasks, use of different architectures for different problems, and inability to scale to complex scenes containing a varying or large number of objects. This project hypothesizes that graph neural networks provide a powerful framework that can encode multiple sensor streams over time to provide robots with rich and scalable representations for multi-object and multi-task perception and manipulation. This project examines a number of extensions to graph neural networks in order to address current limitations for their use in autonomous manipulation. Furthermore this project examines novel ways of leveraging learned graph neural networks for manipulation planning and control in clutter and for multi-step, multi-object manipulation tasks. In order to train these large-scale graph net representations this project will use extremely large scale, physically accurate, photo-realistic simulation. All perceptual and behavior generation techniques developed in this project will be experimentally validated on a set of challenging real-world manipulation tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

为了使机器人在日常生活中充当无处不在的助手，他们必须定期包含涉及许多组成部分的许多物体和物体的环境。当前的机器人研究重点是为孤立的操纵任务提供解决方案，开发了不容易跨任务工作的专门表示。该项目旨在使机器人能够在许多操纵任务中学会从多个传感器中代表和理解世界。具体而言，该项目将检查在杂乱无章的环境中的任务，这些任务需要多个不同的选择和放置操作。该项目将开发适用于机器人助手的自主操纵方法。辅助机器人将对增加老年人和某些退化性疾病的人的生活质量产生次要影响。这些方法还适用于在没有明确对象模型的自然或人为灾难区域的操纵。该项目中开发的工具还可以改善机器人的感知，掌握和多步制造技巧，以便从原始的感知数据中学习强大的表示能力，深层神经网络为您提供了最有前途的框架，以实现关键的感知和推理挑战，并构成了自治机器人操纵的基础。尽管它们成功了，但现有方法的扩展很差，而不是对神经元网络的当前局限性的自主机器人组合，这是由于接受隔离任务的培训，使用不同的架构来解决不同的问题，并且无法扩展到包含各种物体或大量对象的复杂风景。该项目假设图形神经元网络提供了一个强大的框架，可以随着时间的推移编码多个传感器流，以为机器人提供丰富而可扩展的表示，以用于多任务和多任务感知和操作。该项目检查了图形中性网络的许多扩展，以解决其在自动操作中使用的当前限制。此外，该项目还研究了利用学习的图形中性网络的新颖方式，以在混乱中进行操纵计划和控制，并进行多步，多对象的操纵任务。为了训练这些大规模的图表净表示，该项目将使用非常大规模的，物理上精确的照片现实模拟。该项目中开发的所有知觉和行为产生技术将在一组挑战者现实世界中的操纵任务上进行实验验证。该奖项反映了NSF的法定任务，并通过使用该基金会的知识分子优点和更广泛的影响来评估标准。