Predicting Human Grasp Locations on Cup Handles by Using Deep Neural Networks to Infer Heat Signatures from Depth Data
Predicting Human Grasp Locations on Cup Handles by Using Deep Neural Networks to Infer Heat Signatures from Depth Data
复制标题
DOI：
10.1109/icmew.2019.00012
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Yijun Jiang;Sean Banerjee;N. Banerjee
中科院分区：
文献类型：
--
作者：
Yijun Jiang;Sean Banerjee;N. Banerjee
在机器人协助人类与物理物体交互的自动化辅助生活中，机器人面临的一个重要挑战是了解人类可能在哪里抓取物体，以便机器人能够以最可靠的配置将物体呈现给用户。在本文中，我们提出了一种使用编码器-解码器卷积神经网络（CNN）来预测人类抓握杯子手柄位置的方法。我们的工作解决的主要挑战是人手引起的物体遮挡阻碍了抓握位置的直接成像。我们的方法利用了这样的见解：一旦释放物体，由于人体与周围环境之间的温差，手会在物体表面留下热信号。我们的 CNN 学习从传统深度传感器获得的图像作为输入，与使用热像仪成像的抓取位置的热特征作为输出之间的映射。给定一个新杯子的深度图像，我们的方法使用经过训练的网络来预测杯子上的抓取概率分布。使用留一杯方法，我们在从 7 个方向成像的 17 个杯子上获得了 5.67 的平均绝对像素预测误差。
In automated assisted living where a robot assists a human to interact with physical objects, an important challenge is for a robot to understand where humans are likely to grasp objects, so that the robot can present the object to a user in the most tenable configuration. In this paper, we present an approach that uses encoder-decoder convolutional neural networks (CNNs) to predict human grasp location on cup handles. The primary challenge addressed by our work is that object occlusion induced by the human hand prevents direct imaging of grasp location. Our approach uses the insight that once the object is released, the hand leaves a heat signature on the object surface due to the temperature differences between the human body and the ambient environment. Our CNNs learn a mapping between images obtained from traditional depth sensors as input and heat signatures of grasp locations imaged using a thermal camera as output. Given the depth image of a novel cup, our approach uses the trained network to predict the grasp probability distribution over the cup. Using a leave-one-cup-out approach, we obtain a mean absolute pixel-wise prediction error of 5.67 on 17 cups imaged from 7 orientations.