NRI: Collaborative Research: Human-Supervised Perception and Grasping in Clutter

半自动操作的基本组成部分之一是机器人掌握人类操作员指示的物体的能力。在许多任务中，人类和机器人一起工作的自然方式是由人类指出要抓取的物体的大致位置，并由机器人产生实现抓取所需的精确运动。这种核心的“自动抓取”功能对于为残疾人和老年人以及各种军事、警察、太空或水下应用提供辅助操作至关重要。但是，在环境混乱或难以确定人类抓取意图的情况下，实现自动抓取功能可能具有挑战性。在这个结合了两家机构专业知识的合作项目中，pi将解决机器人需要主动探索或“询问”环境的情况，以弄清楚人类想要掌握什么以及机器人应该如何做。为了达到这些目的，pi将研究一种在不确定性下进行规划的改进方法，即信念空间规划。信念空间规划非常适合于抓取的主动定位，因为它是一个单一的框架，算法可以对任务的感知导向和目标导向部分进行推理。pi将使用信念空间规划在用户指定的区域内定位环境中可抓取的几何形状，称为抓取可视性。他们还将探索人类与系统交互的不同方式，以控制抓取。这项工作的应用重点将是辅助操作，即老年人或残疾人操作安装在电动轮椅或踏板车上的辅助机械臂。用户研究将确定目标人群操作该系统的最佳方法。该项目将为pi机构的本科生和高中生提供机会，并将酌情纳入他们所教授的课程。这项研究包含了两个关键的创新，pi期望这将使机器人的抓取更加稳健。一是将信仰空间规划的理念融入到规划过程中，把握规划过程。由于信念空间规划可以推断机器人自身的“信息状态”在未来将如何变化，因此它能够在执行任务的过程中生成获取任务相关信息的计划。第二个创新是一种新的抓取感知方法，该方法将抓取能力几何定位在潜在感兴趣的物体附近。这种把握能力方法不仅对信念空间规划者有帮助，而且pi的初步工作表明，这种方法可以准确且非常快速（10Hz）。最后，还将探讨用户界面与抓取目标位置的不确定性之间的联系，计划将人类行为建模为一个不确定系统，其中隐藏变量描述用户意图。

One of the basic building blocks in semi-autonomous manipulation is the ability for a robot to grasp an object that a human operator indicates. There are many tasks where the natural way for a human and robot to work together is for the human to point out the approximate locations of objects to be grasped and for the robot to generate the precise motions necessary to achieve the grasp. This core "auto-grasp" functionality is critical to providing assistive manipulation for the disabled and elderly, as well as for a variety of military, police, space, or underwater applications. But implementing auto-grasp capability can be challenging in situations where the environment is cluttered, or when it is difficult to determine the grasp intention of the human. In this collaborative project that combines expertise from two institutions, the PIs will tackle situations where it is necessary for the robot to actively explore or "interrogate" the environment in order to figure out what the human intends to grasp and how the robot should do it. To these ends, the PIs will investigate a modified approach to planning under uncertainty known as belief space planning. Belief space planning is well-suited to active localization for grasping, because it is a single framework in which the algorithm can reason about perception-oriented and goal-orientation parts of the task. The PIs will use belief space planning to localize graspable geometries in the environment, known as grasp affordances, in a region indicated by the user. They will also explore different ways in which a human can interact with the system in order to control the grasping. The application focus of the work will be in assistive manipulation, where a person who is elderly or disabled operates an assistive robot arm mounted on an electric wheelchair or scooter. User studies will determine the best methods for the target population to operate the system. The project will contribute to the opportunities available for undergraduates and high school students in the PIs' institutions, and it will also be integrated as appropriate into the curricula of the courses they teach.This research contains two key innovations that the PIs expect will make robot grasping more robust. The first is to incorporate ideas from belief space planning into the reach and grasp planning process. Because belief space planning can reason about how the robot's own "state of information" is expected to change in the future, it is capable of producing plans that acquire task-relevant information in the course of performing a task. The second innovation is a new approach to perception-for-grasping that localizes grasp affordance geometries in the neighborhoods of objects of potential interest. Not only is this grasp affordance approach helpful to the belief space planner, but the PIs' preliminary work indicates that this approach can be accurate and very fast (10Hz). Finally, the connection between the user interface and uncertainty in the location of the grasp target will also be explored, the plan being to model human behavior as an uncertain system where hidden variables describe user intention.