NRI: Collaborative Research: Human-Supervised Perception and Grasping in Clutter

NRI：协作研究：人类监督的杂乱中的感知和抓取

• 批准号: 1427081
• 负责人: Robert Platt
• 金额: $ 75万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-15 至 2020-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1427081&HistoricalAwards=false
• 关键词: NRI; Collaborative; Research; Human; Supervised

One of the basic building blocks in semi-autonomous manipulation is the ability for a robot to grasp an object that a human operator indicates. There are many tasks where the natural way for a human and robot to work together is for the human to point out the approximate locations of objects to be grasped and for the robot to generate the precise motions necessary to achieve the grasp. This core "auto-grasp" functionality is critical to providing assistive manipulation for the disabled and elderly, as well as for a variety of military, police, space, or underwater applications. But implementing auto-grasp capability can be challenging in situations where the environment is cluttered, or when it is difficult to determine the grasp intention of the human. In this collaborative project that combines expertise from two institutions, the PIs will tackle situations where it is necessary for the robot to actively explore or "interrogate" the environment in order to figure out what the human intends to grasp and how the robot should do it. To these ends, the PIs will investigate a modified approach to planning under uncertainty known as belief space planning. Belief space planning is well-suited to active localization for grasping, because it is a single framework in which the algorithm can reason about perception-oriented and goal-orientation parts of the task. The PIs will use belief space planning to localize graspable geometries in the environment, known as grasp affordances, in a region indicated by the user. They will also explore different ways in which a human can interact with the system in order to control the grasping. The application focus of the work will be in assistive manipulation, where a person who is elderly or disabled operates an assistive robot arm mounted on an electric wheelchair or scooter. User studies will determine the best methods for the target population to operate the system. The project will contribute to the opportunities available for undergraduates and high school students in the PIs' institutions, and it will also be integrated as appropriate into the curricula of the courses they teach.This research contains two key innovations that the PIs expect will make robot grasping more robust. The first is to incorporate ideas from belief space planning into the reach and grasp planning process. Because belief space planning can reason about how the robot's own "state of information" is expected to change in the future, it is capable of producing plans that acquire task-relevant information in the course of performing a task. The second innovation is a new approach to perception-for-grasping that localizes grasp affordance geometries in the neighborhoods of objects of potential interest. Not only is this grasp affordance approach helpful to the belief space planner, but the PIs' preliminary work indicates that this approach can be accurate and very fast (10Hz). Finally, the connection between the user interface and uncertainty in the location of the grasp target will also be explored, the plan being to model human behavior as an uncertain system where hidden variables describe user intention.

半自主操作的基本构件之一是机器人抓取人类操作员指示的对象的能力。在许多任务中，人类和机器人合作的自然方式是人类指出要抓取的物体的大致位置，并由机器人产生实现抓取所需的精确运动。这种核心的“自动抓取”功能对于为残疾人和老年人以及各种军事、警察、空间或水下应用提供辅助操作至关重要。但是，在环境混乱的情况下，或者当很难确定人类的抓取意图时，实现自动抓取能力可能是具有挑战性的。在这个将两个机构的专业知识结合在一起的合作项目中，PI将处理机器人必须积极探索或“审问”环境的情况，以便弄清楚人类打算掌握什么以及机器人应该如何做。为此，PI将研究一种在不确定情况下改进的规划方法，称为信念空间规划。信念空间规划非常适合于主动定位抓取，因为它是一个单一的框架，算法可以在其中对任务的面向感知和面向目标的部分进行推理。PI将使用信念空间规划来在用户指示的区域中定位环境中的可抓握几何，称为抓握启示。他们还将探索人类与系统互动以控制抓取的不同方式。这项工作的应用重点将是辅助操纵，即老年人或残疾人操作安装在电动轮椅或滑板车上的辅助机械臂。用户研究将确定目标人群操作该系统的最佳方法。该项目将有助于为大学生和高中生提供在私人投资机构的机会，它也将被适当地整合到他们所教授的课程的课程中。这项研究包含两个关键创新，个人投资机构预计将使机器人抓取更加健壮。第一个是将信念空间规划的想法融入到REACH和把握规划过程中。由于信念空间规划可以推断机器人自身的信息状态在未来预计将如何变化，它能够产生在执行任务过程中获取与任务相关的信息的计划。第二个创新是知觉换抓取的新方法，它将抓取能力几何定位在潜在感兴趣对象的邻近区域。这种抓取启示方法不仅有助于信念空间规划者，而且PI的初步工作表明，这种方法可以非常准确和快速(10赫兹)。最后，还将探索用户界面与抓取目标位置的不确定性之间的联系，计划将人类行为建模为其中隐藏变量描述用户意图的不确定系统。

项目成果

Pick and Place Without Geometric Object Models

无需几何对象模型即可拾取和放置

• DOI: 10.1109/icra.2018.8460553
• 发表时间: 2018
• 期刊: Proceedings of 2018 IEEE International Conference on Robotics and Automation (ICRA
• 作者: Gualtieri, Marcus;Pas, Andreas ten;Platt, Robert
• 通讯作者: Platt, Robert

Online abstraction with MDP homomorphisms for Deep Learning

• 发表时间: 2018-11
• 作者: Ondrej Biza;Robert W. Platt

Towards Assistive Robotic Pick and Place in Open World Environments

在开放世界环境中实现辅助机器人拾取和放置

• 发表时间: 2019
• 期刊: International Symposium on Robotics Research
• 作者: Wang, Dian;Kohler, Colin;ten Pas, Andreas;Wilkinson, Alexander;Liu, Maozhi;Yanco, Holly;Platt, Robert
• 通讯作者: Platt, Robert

Deictic Image Mapping: An Abstraction For Learning Pose Invariant Manipulation Policies

指示图像映射：学习姿势不变操作策略的抽象

• 发表时间: 2019
• 期刊: Proceedings of AAAI Conf. on Artificial Intelligence 2019
• 作者: Platt, Robert;Kohler, Colin;Gualtieri, Marcus
• 通讯作者: Gualtieri, Marcus