Visual Foresight Trees for Object Retrieval From Clutter With Nonprehensile Rearrangement
Visual Foresight Trees for Object Retrieval From Clutter With Nonprehensile Rearrangement
复制标题
DOI：
10.1109/lra.2021.3123373
复制
发表时间：
2022-01-01
影响因子：
5.2
通讯作者：
Boularias, Abdeslam
中科院分区：
文献类型：
--
作者：
Huang, Baichuan;Han, Shuai D.;Boularias, Abdeslam
这封信考虑了从许多紧密堆积的物体中检索物体的问题，使用机器人推动和抓取动作的组合。在密集的杂波中检索物体是机器人在家庭和日常环境中有效操作的重要技能。所提出的视觉预见树（Visual Foresight Tree， VFT）解决方案可以智能地重新排列目标物体周围的杂波，从而使目标物体更容易被捕获。嵌套的不可抓握动作的重排是具有挑战性的，因为它需要在多个对象的组合大配置空间中预测复杂的对象交互。我们首先展示了一个深度神经网络可以被训练来准确地预测当机器人推动其中一个物体时，被包装物体的姿势。该预测网络提供了视觉预见性，并作为场景图像空间中的状态转移函数用于树搜索。树式搜索返回一系列连续的推送操作，这些操作产生用于抓取目标对象的杂波的最佳排列。在几个具有挑战性的任务中，仿真实验和使用真实机器人和物体的实验表明，所提出的方法在成功率和执行动作数量方面优于无模型技术和基于模型的近视方法。介绍VFT和机器人实验的视频可以在https://youtu.be/7cL-hmgvyec上找到。完整的源代码可从https://github.com/arc-l/vft获得。
This letter considers the problem of retrieving an object from many tightly packed objects using a combination of robotic pushing and grasping actions. Object retrieval in dense clutter is an important skill for robots to operate in households and everyday environments effectively. The proposed solution, Visual Foresight Tree (VFT), intelligently rearranges the clutter surrounding a target object so that it can be grasped easily. Rearrangement with nested nonprehensile actions is challenging as it requires predicting complex object interactions in a combinatorially large configuration space of multiple objects. We first show that a deep neural network can be trained to accurately predict the poses of the packed objects when the robot pushes one of them. The predictive network provides visual foresight and is used in a tree search as a state transition function in the space of scene images. The tree search returns a sequence of consecutive push actions yielding the best arrangement of the clutter for grasping the target object. Experiments in simulation and using a real robot and objects show that the proposed approach outperforms model-free techniques as well as model-based myopic methods both in terms of success rates and the number of executed actions, on several challenging tasks. A video introducing VFT, with robot experiments, is accessible at https://youtu.be/7cL-hmgvyec. The full source code is available at https://github.com/arc-l/vft.