Interactive Robot Knowledge Patching Using Augmented Reality
Interactive Robot Knowledge Patching Using Augmented Reality
复制标题
使用增强现实进行交互式机器人知识修补
DOI：
10.1109/icra.2018.8462837
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Song
中科院分区：
文献类型：
--
作者：
Hangxin Liu;Yaofang Zhang;Wenwen Si;Xu Xie;Yixin Zhu;Song
我们通过Microsoft Hololens提出了一种新颖的增强现实方法（AR）方法，以解决诊断，教学和修补机器人可解释知识的挑战性问题。从人的演示中学到了开放瓶的时间和图形（T-AOG），并将其编程到机器人。该表示形式产生了层次结构，该结构捕获给定任务的组成性质，这对用户来说是高度可解释的。通过通过解析T-AOG来可视化由T-AOG代表的知识结构和决策过程，用户可以直观地了解机器人知道的知识，监督机器人的动作计划者并监视视觉上潜在的机器人状态（例如，力量在互动过程中施加）。鉴于一项新任务，通过对机器人内部功能的如此全面的可视化，用户可以快速识别失败的原因，以新的操作交互方式教授机器人，并将其修补为当前的知识结构。通过这种方式，机器人只能通过用户交互方式来解决相似但新任务的求解。这个过程证明了我们的知识表示形式的解释性和AR界面的有效性。
We present a novel Augmented Reality (AR) approach, through Microsoft HoloLens, to address the challenging problems of diagnosing, teaching, and patching interpretable knowledge of a robot. A Temporal And-Or graph (T-AOG) of opening bottles is learned from human demonstration and programmed to the robot. This representation yields a hierarchical structure that captures the compositional nature of the given task, which is highly interpretable for the users. By visualizing the knowledge structure represented by a T-AOG and the decision making process by parsing the T-AOG, the user can intuitively understand what the robot knows, supervise the robot's action planner, and monitor visually latent robot states (e.g., the force exerted during interactions). Given a new task, through such comprehensive visualizations of robot's inner functioning, users can quickly identify the reasons of failures, interactively teach the robot with a new action, and patch it to the current knowledge structure. In this way, the robot is capable of solving similar but new tasks only through minor modifications provided by the users interactively. This process demonstrates the interpretability of our knowledge representation and the effectiveness of the AR interface.
DOI：
10.1109/tpami.2017.2689007
发表时间：
2018-03-01
影响因子：
23.6
作者：
Liu, Xiaobai;Zhao, Yibiao;Zhu, Song-Chun
通讯作者：
Zhu, Song-Chun
影响因子：
56.9
作者：
Brown, Noam;Sandholm, Tuomas
通讯作者：
Sandholm, Tuomas