Visual Reference of Ambiguous Objects for Augmented Reality-Powered Human-Robot Communication in a Shared Workspace
Visual Reference of Ambiguous Objects for Augmented Reality-Powered Human-Robot Communication in a Shared Workspace
复制标题
DOI：
10.1007/978-3-030-49695-1_37
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Peng Gao;Brian Reily;Savannah Paul;Hao Zhang
中科院分区：
文献类型：
--
作者：
Peng Gao;Brian Reily;Savannah Paul;Hao Zhang
在共享工作区中，使用一组公共对象的团队成员必须能够明确地引用单个对象，以便有效地协作。当队友是自主机器人时，人类队友必须能够在不明显干扰他们的工作流程的情况下交流他们预期的参考对象。在人机交互中，视觉参考问题被定义为识别人类所指的特定对象（例如，通过增强现实设备识别的指向手势），并将该对象与机器人队友视野中的相关对象联系起来，从而从一组模糊对象中识别出预期对象。由于人类和机器人团队成员通常从不同的角度观察他们共享的工作空间，因此实现对象的视觉参考是一个具有挑战性但又至关重要的问题。在本文中，我们提出了一种新的模糊对象视觉参考方法，该方法引入了一种基于图匹配的方法，该方法通过增强现实驱动的人机通信融合了共享工作空间中对象的视觉和空间信息。我们的方法用图形表示场景中的对象，其中边编码对象之间的空间关系，属性向量描述与每个节点相关的每个对象的外观。然后，我们将共享工作空间中人机通信的可视化对象引用作为基于优化的图匹配问题，该问题识别基于人和机器人队友观察构建的图中节点的对应关系。我们对引入的两个数据集进行了广泛的实验评估，表明我们的方法能够获得模糊对象的准确视觉参考，并且优于现有的视觉参考方法。
In shared workspaces, teammates working with a common set of objects must be able to unambiguously reference individual objects in order to effectively collaborate. When teammates are autonomous robots, human teammates must be able to communicate their intended reference object without overtly interfering with their workflow. In human-robot interaction, the problem of visual reference is defined as identifying the specific object referred to by a human (e.g., through a pointing gesture recognized by an augmented reality device), and relating this object to the associated object in the robotic teammate’s field of view, thereby identifying the intended object from a set of ambiguous objects. As human and robot teammates typically observe their shared workspace from differing perspectives, achieving visual reference of objects is a challenging yet crucial problem. In this paper, we present a novel approach to visual reference of ambiguous objects that introduces a graph matching-based approach which fuses visual and spatial information of the objects in a shared workspace through augmented reality-powered human-robot communication. Our approach represents the objects in a scene with a graph where edges encoding the spatial relationships among objects and attribute vectors describing each object’s appearance associated with each node. Then, we formulate visual object reference for human-robot communication in a shared workspace as an optimization-based graph matching problem, which identifies the correspondence of nodes in graphs built from the human and robot teammates’ observations. We conduct extensive experimental evaluation on two introduced datasets, showing that our approach is able to obtain accurate visual references of ambiguous objects and outperforms existing visual reference methods.