Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch
Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch
复制标题
DOI：
10.1109/lra.2022.3231520
复制
发表时间：
2022-12
影响因子：
5.2
通讯作者：
Martin Matak;Tucker Hermans
中科院分区：
文献类型：
--
作者：
Martin Matak;Tucker Hermans
可靠地规划多指手的指尖抓握对于许多任务（包括工具使用、插入和灵巧的手部操作）来说是一个关键挑战。当机器人缺乏要抓取的物体的精确模型时，这项任务变得更加困难。触觉传感提供了一种有前景的方法来解释物体形状的不确定性。然而，当前的机械手往往缺乏完整的触觉覆盖。因此，出现了如何计划和执行多指手的抓握以便与触觉传感器覆盖的区域进行接触的问题。为了解决这个问题，我们提出了一种抓取规划的方法，该方法明确地解释指尖应该接触估计的物体表面的位置，同时最大限度地提高抓取成功的概率。我们的方法成功的关键是使用视觉表面估计进行初始规划以编码接触约束。然后，机器人使用触觉反馈控制器执行该计划，该控制器使机器人能够适应物体表面的在线估计，以纠正初始计划中的错误。重要的是，机器人从未明确整合视觉和触觉感知之间的物体姿态或表面估计，而是以互补的方式使用这两种模式。视觉在接触之前引导机器人运动；当接触发生与视觉预测不同时，触摸会更新计划。我们表明，我们的方法使用单个摄像机视图的表面估计成功地综合并执行了对以前未见过的物体的精确抓取。此外，我们的方法优于最先进的多指抓取规划器，同时也超过了我们提出的几个基线。
Reliably planning fingertip grasps for multi-fingered hands lies as a key challenge for many tasks including tool use, insertion, and dexterous in-hand manipulation. This task becomes even more difficult when the robot lacks an accurate model of the object to be grasped. Tactile sensing offers a promising approach to account for uncertainties in object shape. However, current robotic hands tend to lack full tactile coverage. As such, a problem arises of how to plan and execute grasps for multi-fingered hands such that contact is made with the area covered by the tactile sensors. To address this issue, we propose an approach to grasp planning that explicitly reasons about where the fingertips should contact the estimated object surface while maximizing the probability of grasp success. Key to our method's success is the use of visual surface estimation for initial planning to encode the contact constraint. The robot then executes this plan using a tactile-feedback controller that enables the robot to adapt to online estimates of the object's surface to correct for errors in the initial plan. Importantly, the robot never explicitly integrates object pose or surface estimates between visual and tactile sensing, instead it uses the two modalities in complementary ways. Vision guides the robots motion prior to contact; touch updates the plan when contact occurs differently than predicted from vision. We show that our method successfully synthesises and executes precision grasps for previously unseen objects using surface estimates from a single camera view. Further, our approach outperforms a state of the art multi-fingered grasp planner, while also beating several baselines we propose.