Human–Robot Collaborative Control in a Virtual-Reality-Based Telepresence System
Human–Robot Collaborative Control in a Virtual-Reality-Based Telepresence System
复制标题
基于虚拟现实的远程呈现系统中的人机协作控制
DOI：
10.1007/s12369-020-00718-w
复制
发表时间：
2020
影响因子：
4.7
通讯作者：
Sheng, Weihua
中科院分区：
文献类型：
--
作者：
Du, Jianhao;Do, Ha Manh;Sheng, Weihua
在本文中，我们开发了一个机器人远程呈现系统，通过定制设计的移动的机器人提供远程用户在本地环境中的沉浸式体现。建议的远程呈现系统使用虚拟现实（VR）设备连接到机器人的远程用户。来自RGB-D摄像头的三维视觉数据在VR设备中进行实时立体显示，形成深度耦合的人机系统，并创建身临其境的远程呈现体验。基于用户的研究，它被发现，更好的用户体验，可以通过允许机器人跟踪扬声器，同时意识到远程用户的意图。为此，我们提出了一个基于人的意图识别和声音定位的人-机器人协同控制框架。使用隐马尔可夫模型基于VR设备的运动来推断远程用户的头部移动的意图。使用麦克风阵列通过声源定位来跟踪说话者。提出了一种融合机器人和远程用户控制的协同控制方案。实验在一对一和一对二远程会话的情况下进行。实验结果表明，该系统能够显著提高机器人临场感系统的沉浸感和性能，从而极大地提升了机器人临场感系统的用户体验。
In this paper, we develop a robotic telepresence system to provide remote users with immersive embodiment in local environments through a custom-designed mobile robot. The proposed telepresence system uses a virtual reality (VR) device to connect a remote user to the robot. Three dimensional visual data from a RGB-D camera are rendered for real-time stereoscopic display in the VR device, which forms a deeply-coupled human machine system and creates an immersive experience of telepresence. Based on a user study, it is found that better user experience can be achieved by allowing the robot to track the speaker while being aware of the intention of the remote user. To this end we propose a human-robot collaborative control framework based on human intention recognition and sound localization. The intentions of head movement of the remote user are inferred based on the motion of the VR device using hidden Markov models. The speaker is tracked through sound source localization using a microphone array. A collaborative control scheme is developed to fuse the control from the robot and the remote user. Experiments are conducted in both one-to-one and one-to-two remote conversation scenarios. The results show that the proposed system can significantly improve the immersiveness and performance of robotic telepresence systems, therefore greatly enhancing the user experience of such telepresence systems.
辅助移动机器人远程操作的上下文任务感知共享自主权
DOI：
10.1109/iros.2014.6943023
发表时间：
2014
期刊：
2014 IEEE/RSJ International Conference on Intelligent Robots and Systems
影响因子：
--
作者：
Ming Gao;Jan Oberländer;T. Schamm;Johann Marius Zöllner
通讯作者：
Johann Marius Zöllner
社交远程体现：理解存在
DOI：
--
发表时间：
2001
期刊：
Auton. Robots
影响因子：
--
作者：
E. Paulos;J. Canny
通讯作者：
J. Canny
动态环境的机器人试镜
DOI：
--
发表时间：
2012
期刊：
International Conference on Signal Processing, Communications and Computing
影响因子：
--
作者：
K. Nakadai;G. Ince;K. Nakamura;H. Nakajima
通讯作者：
H. Nakajima