Sketching Affordances for Human-in-the-loop Robotic Manipulation Tasks
Sketching Affordances for Human-in-the-loop Robotic Manipulation Tasks
复制标题
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Sina Masnadi;J. Laviola;Jana Pavlasek;Xiaofang Zhu;Karthik Desingh;O. C. Jenkins
中科院分区：
文献类型：
--
作者：
Sina Masnadi;J. Laviola;Jana Pavlasek;Xiaofang Zhu;Karthik Desingh;O. C. Jenkins
为了让机器人在自然的人类环境中执行复杂的自主操作任务，它们必须能够在各种物体上进行灵活的动作，而不是挑选和放置。给定代理在一组对象上可用的操作称为对象负担。启示的概念最早是由心理学家J·J·吉布森[1]提出的，近年来引起了机器人学界的极大兴趣[2]、[3]。如何最好地获得这些负担在机器人学中是一个悬而未决的问题。利用人类对物品负担的知识，机器人可以与各种常见的家用物品互动，以执行复杂的任务，如煮一杯咖啡。我们建议让人类用户在没有机器人学和编程方面的专业知识的情况下，将关于给定场景中的负担能力的知识传递给机器人。为此，我们提出了一个易于使用的系统，通过在图形界面上绘制草图来获取对象几何形状及其关联的启示。这允许用户通过利用基于草图的技术来与机器人系统交互，以提供直观的用户界面，如图2所示。用户绘制对象的几何图形及其承受能力。在任务执行过程中，当机器人遇到其具有启示信息的对象时，它可以通过将对象几何注册到其RGB-D数据中来执行启示，然后顺序地执行动作以实现目标。绘制草图是一种与机器人快速沟通的低开销方法，它可以快速提供有关环境和机器人可用的动作的信息。它为用户提供了以启示模板的形式向机器人快速演示看不见的任务的能力。我们的系统支持人在回路中的方法，使机器人能够在不同的对象上执行各种操作任务。获取物体的三维网格模型可以帮助估计物体的姿态，但许多用于这项任务的方法[5]-[7]依赖于综合设计的网格模型。在杂乱的场景中，在机器人的RGB-D数据上绘制草图可以提供精确的几何图形，而开销很小。Maghoumi等人。介绍了一个基于草图的系统，用于从杂乱场景中的点云中提取3D几何图形，并恢复
In order for robots to perform complex autonomous manipulation tasks in natural human environments, they must be able to carry out dexterous actions, beyond pick-andplace, on a wide variety of objects. The actions which are available to a given agent over a set of objects are called object affordances. The concept of affordance was first introduced by psychologist J. J. Gibson [1] and has garnered much interest in the robotics community in recent years [2], [3]. How to best acquire these affordances is an open problem in robotics. Leveraging human knowledge of object affordances can enable robots to interact with a variety of common household objects to perform complex tasks, such as making a cup of coffee. We propose to enable a human user, without expert knowledge about robotics and programming, to transfer knowledge about affordances in a given scene to a robot. To this end, we propose an easy-to-use system to acquire object geometries and their associated affordances through sketching on a graphical interface. This allows users to interact with robotic systems by utilizing sketch-based techniques to provide a straightforward user interface, as shown in Figure 2. The user sketches the geometry of the object and its affordances. During task execution, when the robot encounters the objects for which it has affordance information, it can execute the affordances by registering the object geometries to its RGB-D data and then performing actions sequentially to achieve the goal. Sketching is a low overhead means of rapid communication with the robot which can quickly provide information about the environment and the actions available to the robot. It provides the user with the ability to quickly demonstrate unseen tasks to the robot in the form of affordance templates [4]. Our system facilitates a human-in-the-loop approach which enables the robot to perform a large variety of manipulation tasks on different objects. Access to object 3D mesh models can assist in object pose estimation, but many methods for this task [5]–[7] rely on synthetically designed mesh models. In cluttered scenes, sketching over RGB-D data from the robot can provide precise geometries with very little overhead. Maghoumi et al. introduced a sketch-based system to extract 3D geometries from point clouds in cluttered scenes which recovers