Unseen Object Instance Segmentation for Robotic Environments
Unseen Object Instance Segmentation for Robotic Environments
复制标题
DOI：
10.1109/tro.2021.3060341
复制
发表时间：
2020-07
影响因子：
7.8
通讯作者：
Christopher Xie;Yu Xiang;Arsalan Mousavian;D. Fox
中科院分区：
文献类型：
--
作者：
Christopher Xie;Yu Xiang;Arsalan Mousavian;D. Fox
为了在非结构化环境中发挥作用，机器人需要具有识别看不见的物体的能力。我们通过解决桌面环境中看不见的对象实例的分割问题，朝这个方向迈出了一步。然而，对于大多数机器人设置来说，此任务所需的大规模现实世界数据集类型通常不存在，这促使了合成数据的使用。我们提出的方法，看不见的对象实例分割（UOIS）-Net，分别利用合成 RGB 和合成深度来进行看不见的对象实例分割。 UOIS-Net 由两个阶段组成：首先，它仅在深度上运行，以产生 2D 或 3D 的对象实例中心投票，并将它们组装成粗略的初始掩模。其次，这些初始蒙版使用 RGB 进行细化。令人惊讶的是，我们的框架能够从合成 RGB-D 数据中学习，其中 RGB 是非真实感的。为了训练我们的方法，我们引入了桌面上随机对象的大规模合成数据集。我们表明，我们的方法可以产生清晰而准确的分割掩模，在不可见的对象实例分割方面优于最先进的方法。我们还表明，我们的方法可以分割看不见的物体以供机器人抓取。
In order to function in unstructured environments, robots need the ability to recognize unseen objects. We take a step in this direction by tackling the problem of segmenting unseen object instances in tabletop environments. However, the type of large-scale real-world dataset required for this task typically does not exist for most robotic settings, which motivates the use of synthetic data. Our proposed method, unseen object instance segmentation (UOIS)-Net, separately leverages synthetic RGB and synthetic depth for unseen object instance segmentation. UOIS-Net is composed of two stages: first, it operates only on depth to produce object instance center votes in 2D or 3D and assembles them into rough initial masks. Second, these initial masks are refined using RGB. Surprisingly, our framework is able to learn from synthetic RGB-D data where the RGB is nonphotorealistic. To train our method, we introduce a large-scale synthetic dataset of random objects on tabletops. We show that our method can produce sharp and accurate segmentation masks, outperforming state-of-the-art methods on unseen object instance segmentation. We also show that our method can segment unseen objects for robot grasping.