Vis2Hap: Vision-based Haptic Rendering by Cross-modal Generation
Vis2Hap: Vision-based Haptic Rendering by Cross-modal Generation
复制标题
DOI：
10.1109/icra48891.2023.10160373
复制
发表时间：
2023-01
期刊：
影响因子：
--
通讯作者：
G. Cao;Jiaqi Jiang;N. Mao;D. Bollegala;Min Li;Shan Luo
中科院分区：
文献类型：
--
作者：
G. Cao;Jiaqi Jiang;N. Mao;D. Bollegala;Min Li;Shan Luo
为了协助机器人完成远程操作任务，近年来开发了触觉渲染，它允许人类操作员获得虚拟的触摸感觉。大多数以前的触觉渲染方法强烈依赖于触觉传感器收集的数据。然而，由于机器人可触及的空间有限以及触觉传感器的限制，触觉数据并未广泛用于机器人。为了消除对触觉数据的需求，在本文中，我们提出了一种名为 Vis2Hap 的新颖方法，用于从视觉输入生成触觉渲染，这些视觉输入可以从远处获得，无需物理交互。我们将物体的表面纹理作为传达给操作人员的关键线索。为此，设计了生成模型来模拟物体表面的粗糙度和光滑度。为了在 Vis2Hap 中嵌入触觉提示，我们使用触觉传感器的高度图和摩擦系数的频谱图作为生成模型的中间输出。 Vis2Hap 经过训练后，可用于生成新表面纹理的高度图和频谱图，从中可以获得摩擦图像并将其显示在触觉显示器上。用户研究表明，我们提出的 Vis2Hap 方法使用户能够获得类似于物理对象的真实触觉感觉。所提出的基于视觉的触觉渲染有可能增强人类操作员对远程环境的感知并促进机器人操作。
To assist robots in teleoperation tasks, haptic rendering which allows human operators access a virtual touch feeling has been developed in recent years. Most previous haptic rendering methods strongly rely on data collected by tactile sensors. However, tactile data is not widely available for robots due to their limited reachable space and the restrictions of tactile sensors. To eliminate the need for tactile data, in this paper we propose a novel method named as Vis2Hap to generate haptic rendering from visual inputs that can be obtained from a distance without physical interaction. We take the surface texture of objects as key cues to be conveyed to the human operator. To this end, a generative model is designed to simulate the roughness and slipperiness of the object's surface. To embed haptic cues in Vis2Hap, we use height maps from tactile sensors and spectrograms from friction coefficients as the intermediate outputs of the generative model. Once Vis2Hap is trained, it can be used to generate height maps and spectrograms of new surface textures, from which a friction image can be obtained and displayed on a haptic display. The user study demonstrates that our proposed Vis2Hap method enables users to access a realistic haptic feeling similar to that of physical objects. The proposed vision-based haptic rendering has the potential to enhance human operators' perception of the remote environment and facilitate robotic manipulation.