IOSG: Image-Driven Object Searching and Grasping
IOSG: Image-Driven Object Searching and Grasping
复制标题
DOI：
10.1109/iros55552.2023.10342009
复制
发表时间：
2023-08
期刊：
影响因子：
--
通讯作者：
Houjian Yu;Xibai Lou;Yang Yang-Yang;Changhyun Choi
中科院分区：
文献类型：
--
作者：
Houjian Yu;Xibai Lou;Yang Yang-Yang;Changhyun Choi
当机器人从杂乱的场景中检索特定对象时，例如家庭和仓库环境，目标对象通常被部分遮挡或完全隐藏。因此，机器人需要搜索、识别目标对象并成功抓取。之前的工作依赖于预先训练的对象识别或分割模型来找到目标对象。然而，这种方法需要费力的手动注释来训练模型，甚至无法找到新的目标对象。在本文中，我们提出了一个图像驱动的对象搜索和抓取（IOSG）的方法，机器人提供了一个新的目标对象的参考图像，并负责查找和检索它。我们设计了一个目标相似性网络，生成一个概率图来推断新目标的位置。IOSG学习分层策略;高级策略预测子任务类型，而低级策略（资源管理器和协调器）生成有效的推送和抓取操作。浏览器负责在目标对象被其他对象隐藏或遮挡时搜索目标对象。一旦目标物体被发现，协调器进行面向目标的推动和抓取，以从杂乱中检索目标。所提出的管道在仿真中进行了完全自我监督的训练，并应用于真实的环境。我们的模型实现了96.0%和94.5%的任务成功率分别协调和探索任务的仿真，和85.0%的成功率为真实的机器人的搜索和抓取任务。请参阅我们的项目页面了解更多信息：https://z.umn.edu/iosg。
When robots retrieve specific objects from cluttered scenes, such as home and warehouse environments, the target objects are often partially occluded or completely hidden. Robots are thus required to search, identify a target object, and successfully grasp it. Preceding works have relied on pre-trained object recognition or segmentation models to find the target object. However, such methods require laborious manual annotations to train the models and even fail to find novel target objects. In this paper, we propose an Image-driven Object Searching and Grasping (IOSG) approach where a robot is provided with the reference image of a novel target object and tasked to find and retrieve it. We design a Target Similarity Network that generates a probability map to infer the location of the novel target. IOSG learns a hierarchical policy; the high-level policy predicts the subtask type, whereas the low-level policies, explorer and coordinator, generate effective push and grasp actions. The explorer is responsible for searching the target object when it is hidden or occluded by other objects. Once the target object is found, the coordinator conducts target-oriented pushing and grasping to retrieve the target from the clutter. The proposed pipeline is trained with full self-supervision in simulation and applied to a real environment. Our model achieves a 96.0% and 94.5% task success rate on coordination and exploration tasks in simulation respectively, and 85.0% success rate on a real robot for the search-and-grasp task. Please refer to our project page for more information: https://z.umn.edu/iosg.