Generation of GelSight Tactile Images for Sim2Real Learning
Generation of GelSight Tactile Images for Sim2Real Learning
复制标题
DOI：
10.1109/lra.2021.3063925
复制
发表时间：
2021-04-01
影响因子：
5.2
通讯作者：
Luo, Shan
中科院分区：
文献类型：
--
作者：
Gomes, Daniel Fernandes;Paoletti, Paolo;Luo, Shan
在机器人操作任务的模拟到现实（Sim2Real）学习中，目前大多数工作都利用了相机视觉，但在操作过程中，相机视觉可能会被机器人的手严重遮挡。触觉传感为视觉提供了补充信息，可以弥补因遮挡造成的信息损失。然而，由于没有可用的模拟触觉传感器，触觉传感在模拟到现实的研究中受到限制。为了缩小这一差距，我们引入了一种在常用的Gazebo模拟器中模拟GelSight触觉传感器的新方法。与真实的GelSight传感器类似，模拟传感器可以从模拟光学传感器捕获的深度图生成高分辨率图像，并重建被触摸物体与不透明软膜之间的相互作用。它可以间接感知物体的力、几何形状、纹理和其他属性，并实现带有触觉传感的模拟到现实学习。初步实验结果表明，模拟传感器可以产生与真实GelSight传感器捕获的结果相似的逼真输出。本文所使用的所有材料都可以在https://danfergo.github.io/gelsight - simulation获取。
Most current works in Sim2Real learning for robotic manipulation tasks leverage camera vision that may be significantly occluded by robot hands during the manipulation. Tactile sensing offers complementary information to vision and can compensate for the information loss caused by the occlusions. However, the use of tactile sensing is restricted in the Sim2Real research due to no simulated tactile sensors being available. To mitigate the gap, we introduce a novel approach for simulating a GelSight tactile sensor in the commonly used Gazebo simulator. Similar to the real GelSight sensor, the simulated sensor can produce high-resolution images from depth-maps captured by a simulated optical sensor, and reconstruct the interaction between the touched object and an opaque soft membrane. It can indirectly sense forces, geometry, texture and other properties of the object and enables Sim2Real learning with tactile sensing. Preliminary experimental results have shown that the simulated sensor could generate realistic outputs similar to the ones captured by a real GelSight sensor. All the materials used in this letter are available at https://danfergo.github.io/gelsight-simulation.