Using Human Gaze to Improve Robustness Against Irrelevant Objects in Robot Manipulation Tasks
Using Human Gaze to Improve Robustness Against Irrelevant Objects in Robot Manipulation Tasks
复制标题
DOI：
10.1109/lra.2020.2998410
复制
发表时间：
2020-05
影响因子：
5.2
通讯作者：
Heecheol Kim;Y. Ohmura;Y. Kuniyoshi
中科院分区：
文献类型：
--
作者：
Heecheol Kim;Y. Ohmura;Y. Kuniyoshi
深度模仿学习可以从原始像素输入中学习复杂的视觉技能。然而，这种方法存在过拟合训练图像的问题。神经网络很容易被与任务无关的对象分心。在这封信中，我们使用头戴式眼动跟踪设备测量的人类凝视来丢弃与任务无关的视觉干扰。我们提出了一种基于混合密度网络的行为克隆方法，学习模仿人类的目光。该模型从原始像素图像预测凝视位置，并在预测的凝视周围裁剪图像。只有这些裁剪的图像用于计算输出动作。这种裁剪过程可以消除视觉干扰，因为视线很少固定在与任务无关的物体上。这种对不相关对象的鲁棒性可以提高机器人在任务不相关对象存在的情况下的操作性能。我们评估了我们的模型上的四个操作任务，旨在测试模型的鲁棒性不相关的对象。实验结果表明，该模型能够从注视点位置预测任务相关对象的位置，对任务无关对象具有较强的鲁棒性，尤其在多对象处理中表现出令人印象深刻的操作性能.
Deep imitation learning enables the learning of complex visuomotor skills from raw pixel inputs. However, this approach suffers from the problem of overfitting to the training images. The neural network can easily be distracted by task-irrelevant objects. In this letter, we use the human gaze measured by a head-mounted eye tracking device to discard task-irrelevant visual distractions. We propose a mixture density network-based behavior cloning method that learns to imitate the human gaze. The model predicts gaze positions from raw pixel images and crops images around the predicted gazes. Only these cropped images are used to compute the output action. This cropping procedure can remove visual distractions because the gaze is rarely fixated on task-irrelevant objects. This robustness against irrelevant objects can improve the manipulation performance of robots in scenarios where task-irrelevant objects are present. We evaluated our model on four manipulation tasks designed to test the robustness of the model to irrelevant objects. The results indicate that the proposed model can predict the locations of task-relevant objects from gaze positions, is robust to task-irrelevant objects, and exhibits impressive manipulation performance especially in multi-object handling.