Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection
复制标题
DOI：
10.1177/0278364917710318
复制
发表时间：
2018-04-01
影响因子：
9.2
通讯作者：
Quillen, Deirdre
中科院分区：
文献类型：
--
作者：
Levine, Sergey;Pastor, Peter;Quillen, Deirdre
我们描述了一种基于学习的方法，从单目图像的机器人抓取手眼协调。为了学习手眼协调抓取，我们训练了一个大型卷积神经网络来预测抓取器的任务空间运动将导致成功抓取的概率，仅使用独立于相机校准或当前机器人姿势的单目相机图像。这需要网络观察场景中抓手和物体之间的空间关系，从而学习手眼协调。然后，我们使用这个网络伺服夹持器在真实的时间，以实现成功的把握。我们描述了两个大规模的实验，我们进行了两个独立的机器人平台。在第一个实验中，在两个月的时间里收集了大约800，000次抓取尝试，在任何给定时间使用6到14个机器人操纵器，相机放置和抓取器磨损的差异。在第二个实验中，我们使用了一个不同的机器人平台和8个机器人来收集一个由超过90万次抓取尝试组成的数据集。第二个机器人平台用于测试机器人之间的传输，以及来自不同机器人组的数据可用于辅助学习的程度。我们的实验结果表明，我们的方法实现了有效的实时控制，可以成功地抓住新的对象，并通过连续伺服纠正错误。我们的转移实验还表明，来自不同机器人的数据可以结合起来，学习更可靠和有效的抓取。
We describe a learning-based approach to hand-eye coordination for robotic grasping from monocular images. To learn hand-eye coordination for grasping, we trained a large convolutional neural network to predict the probability that task-space motion of the gripper will result in successful grasps, using only monocular camera images independent of camera calibration or the current robot pose. This requires the network to observe the spatial relationship between the gripper and objects in the scene, thus learning hand-eye coordination. We then use this network to servo the gripper in real time to achieve successful grasps. We describe two large-scale experiments that we conducted on two separate robotic platforms. In the first experiment, about 800,000 grasp attempts were collected over the course of two months, using between 6 and 14 robotic manipulators at any given time, with differences in camera placement and gripper wear and tear. In the second experiment, we used a different robotic platform and 8 robots to collect a dataset consisting of over 900,000 grasp attempts. The second robotic platform was used to test transfer between robots, and the degree to which data from a different set of robots can be used to aid learning. Our experimental results demonstrate that our approach achieves effective real-time control, can successfully grasp novel objects, and corrects mistakes by continuous servoing. Our transfer experiment also illustrates that data from different robots can be combined to learn more reliable and effective grasping.