Research on Construction of a Flexible Human-Robot Interface

我们开发了一个人机界面系统PARTNER，它考虑到了增强现实方法的灵活性。该原型由用于信息显示的投影子系统和用于识别人体动作的实时跟踪视觉子系统组成。在这个项目中，我们开发了两个新的功能，交互式指针和自适应注释功能。交互式指针用于通过操作员的手势选择环境中的对象或位置。该系统可以直观地跟踪操作员的指示手，并使用LCD投影仪在指定位置投射一个标记。由于可以在实际工作空间中直接观察到标记，而无需显示器或hmd，因此通过移动手来纠正指示位置对操作员来说非常容易。自适应标注功能是为教学和辅助人类任务而开发的。在状态转换图的基础上，系统通过监测物体几何形状的变化和操作人员完成任务所需的动作，自适应地生成标注。实验结果证明了该系统的有效性。此外，我们开发了一种基于视图的视觉跟踪系统，不仅可以应对三维环境中模板外观的变化，还可以解决人与机器人合作处理物体时遇到的遮挡问题。我们提出了基于评估镶嵌模板中的相关误差的镶嵌模板方法来检测遮挡。人与机器人的交接动作实验证明了视觉跟踪系统的有效性。

We have developed a human-robot interface system, PARTNER, that takes into account the flexibility of the Augmented Reality approach. The prototype consists of a projector subsystem for information display and a real-time tracking vision subsystem to recognize the human's action. In this project, we have developed two new functions, Interactive Hand Pointer and Adaptive Annotation Function.The Interactive Hand Pointer is used for selecting objects or positions in the environment via the operator's hand gestures. The system visually tracks the operator's pointing hand and projects a mark at the indicated position using an LCD projector. Since the mark can be observed directly in the real work space without monitor displays or HMDs, correction of the indicated position by moving the hand is very easy for the operator. The Adaptive Annotation, Function has been developed for teaching and assisting the human's task. Based on a state transition diagram, the system generates annotations adaptively to the situation by monitoring not only changes of the object's geometry but also the operator's action required for achieving the task. Experimental results of an unfolding task of a portable OHP device demonstrate the usefulness of the proposed system.Moreover, we have developed a view-based visual tracking system not only to cope with change of the appearance of the template in a 3D environment but also to cope with occlusion which is a problem in cooperatively handling objects between humans and robots. We proposed tessellated template method" to detect occlusion based on an evaluation of correlation errors in the tessellated template. Experiments of hand-over action between human and robot demonstrate usefulness of the visual tracking system.