Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping
Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping
复制标题
DOI：
10.1109/tmech.2022.3201057
复制
发表时间：
2022-08
期刊：
影响因子：
--
通讯作者：
Jiaqi Jiang;G. Cao;Aaron Butterworth;Thanh-Toan Do;Shan Luo
中科院分区：
文献类型：
--
作者：
Jiaqi Jiang;G. Cao;Aaron Butterworth;Thanh-Toan Do;Shan Luo
对于机器人来说，拾取透明物体仍然是一项具有挑战性的任务。透明物体的反射和折射等视觉特性使得目前依靠摄像机感知的抓取方法无法对其进行检测和定位。然而，人类可以很好地处理透明物体，首先观察其粗糙的轮廓，然后戳感兴趣的区域以获得用于抓握的精细轮廓。受此启发，我们提出了一个新的框架，视觉引导的触觉戳透明物体的把握。在所提出的框架中，首先使用分割网络来预测被命名为戳区域的水平上部区域，机器人可以戳物体以获得良好的触觉阅读，同时导致对物体状态的最小干扰。然后使用高分辨率GelSight触觉传感器进行戳。给定局部轮廓与触觉阅读的改善，启发式抓取计划用于抓取透明对象。为了减轻现实世界的数据收集和透明对象的标记的限制，构建了一个大规模的真实感合成数据集。大量的实验表明，我们提出的分割网络可以预测潜在的戳区域与高的平均平均精度（mAP）为0.360，和视觉引导的触觉戳可以提高抓持成功率从38.9%到85.2%显着。由于其简单性，我们提出的方法也可以被其他力或触觉传感器采用，并可用于抓取其他具有挑战性的物体。
Picking up transparent objects is still a challenging task for robots. The visual properties of transparent objects such as reflection and refraction make the current grasping methods that rely on camera sensing fail to detect and localize them. However, humans can handle the transparent object well by first observing its coarse profile and then poking an area of interest to get a fine profile for grasping. Inspired by this, we propose a novel framework of vision-guided tactile poking for transparent objects grasping. In the proposed framework, a segmentation network is first used to predict the horizontal upper regions named as poking regions, where the robot can poke the object to obtain a good tactile reading, while leading to minimal disturbance to the object's state. A poke is then performed with a high-resolution GelSight tactile sensor. Given the local profiles improved with the tactile reading, a heuristic grasp is planned for grasping the transparent object. To mitigate the limitations of real-world data collection and labeling for transparent objects, a large-scale realistic synthetic dataset was constructed. Extensive experiments demonstrate that our proposed segmentation network can predict the potential poking region with a high mean average precision (mAP) of 0.360, and the vision-guided tactile poking can enhance the grasping success rate significantly from 38.9% to 85.2%. Thanks to its simplicity, our proposed approach could also be adopted by other force or tactile sensors and could be used for grasping of other challenging objects.