NRI: Collaborative Research: Human-Supervised Perception and Grasping in Clutter

半自主操作中的基本构建块之一是机器人抓住人类操作员指示的对象的能力。 在许多任务中，人类和机器人一起工作的自然方式是人类指出要抓取的物体的大致位置，机器人生成实现抓取所需的精确运动。 这种核心的“自动抓取”功能对于为残疾人和老年人以及各种军事、警察、太空或水下应用提供辅助操作至关重要。但是，在环境杂乱的情况下，或者当难以确定人类的抓握意图时，实现自动抓握能力可能具有挑战性。 在这个结合了两个机构的专业知识的合作项目中，PI将解决机器人需要主动探索或“询问”环境的情况，以确定人类想要抓住什么以及机器人应该如何做。为此，PI将研究一种改进的方法来在不确定性下进行规划，称为信念空间规划。 信念空间规划非常适合主动定位抓取，因为它是一个单一的框架，在这个框架中，算法可以推理任务的感知导向和目标导向部分。 PI将使用信念空间规划来定位环境中的可抓握几何形状，称为抓握启示，在用户指示的区域中。 他们还将探索人类与系统交互以控制抓取的不同方式。 这项工作的应用重点将是辅助操作，老年人或残疾人操作安装在电动轮椅或踏板车上的辅助机器人手臂。 用户研究将确定目标人群操作该系统的最佳方法。 该项目将为PI机构的本科生和高中生提供机会，并将酌情纳入他们教授的课程。该研究包含两项关键创新，PI预计这将使机器人抓取更加强大。 第一个是将信念空间规划的思想融入到伸手可及的规划过程中。 由于信念空间规划可以推理机器人自己的“信息状态”在未来的预期变化，因此它能够产生在执行任务的过程中获取任务相关信息的计划。 第二个创新是一种新的方法来感知抓取，本地化把握启示几何的潜在利益的对象的邻域。 这种把握启示的方法不仅对信念空间规划者有帮助，而且PI的初步工作表明这种方法可以非常准确和快速（10 Hz）。 最后，用户界面和不确定性之间的联系，把握目标的位置也将进行探讨，该计划是一个不确定的系统，隐藏的变量描述用户的意图来模拟人类的行为。

One of the basic building blocks in semi-autonomous manipulation is the ability for a robot to grasp an object that a human operator indicates. There are many tasks where the natural way for a human and robot to work together is for the human to point out the approximate locations of objects to be grasped and for the robot to generate the precise motions necessary to achieve the grasp. This core "auto-grasp" functionality is critical to providing assistive manipulation for the disabled and elderly, as well as for a variety of military, police, space, or underwater applications. But implementing auto-grasp capability can be challenging in situations where the environment is cluttered, or when it is difficult to determine the grasp intention of the human. In this collaborative project that combines expertise from two institutions, the PIs will tackle situations where it is necessary for the robot to actively explore or "interrogate" the environment in order to figure out what the human intends to grasp and how the robot should do it. To these ends, the PIs will investigate a modified approach to planning under uncertainty known as belief space planning. Belief space planning is well-suited to active localization for grasping, because it is a single framework in which the algorithm can reason about perception-oriented and goal-orientation parts of the task. The PIs will use belief space planning to localize graspable geometries in the environment, known as grasp affordances, in a region indicated by the user. They will also explore different ways in which a human can interact with the system in order to control the grasping. The application focus of the work will be in assistive manipulation, where a person who is elderly or disabled operates an assistive robot arm mounted on an electric wheelchair or scooter. User studies will determine the best methods for the target population to operate the system. The project will contribute to the opportunities available for undergraduates and high school students in the PIs' institutions, and it will also be integrated as appropriate into the curricula of the courses they teach.This research contains two key innovations that the PIs expect will make robot grasping more robust. The first is to incorporate ideas from belief space planning into the reach and grasp planning process. Because belief space planning can reason about how the robot's own "state of information" is expected to change in the future, it is capable of producing plans that acquire task-relevant information in the course of performing a task. The second innovation is a new approach to perception-for-grasping that localizes grasp affordance geometries in the neighborhoods of objects of potential interest. Not only is this grasp affordance approach helpful to the belief space planner, but the PIs' preliminary work indicates that this approach can be accurate and very fast (10Hz). Finally, the connection between the user interface and uncertainty in the location of the grasp target will also be explored, the plan being to model human behavior as an uncertain system where hidden variables describe user intention.