TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baseline
TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baseline
复制标题
DOI：
10.1109/lra.2022.3183256
复制
发表时间：
2022-07-01
影响因子：
5.2
通讯作者：
Lu, Cewu
中科院分区：
文献类型：
--
作者：
Fang, Hongjie;Fang, Hao-Shu;Lu, Cewu
透明物体在我们的日常生活中很常见，经常在自动化生产线上处理。鲁棒的基于视觉的机器人抓取和操纵这些对象将有利于自动化。然而，大多数当前的抓取算法在这种情况下会失败，因为它们严重依赖于深度图像，而普通的深度传感器通常由于光的反射和折射而无法为透明物体产生准确的深度信息。在这封信中，我们通过贡献一个用于透明对象深度完成的大规模真实世界数据集来解决这个问题，该数据集包含来自130个不同场景的57，715个RGB-D图像。我们的数据集是第一个大规模的真实世界数据集，它在多样化和混乱的场景中提供地面真实深度，表面法线，透明遮罩。跨域实验表明，我们的数据集更一般，可以使模型具有更好的泛化能力。此外，我们提出了一个端到端的深度补全网络，它将RGB图像和不准确的深度图作为输入，并输出一个细化的深度图。实验结果表明，该方法具有上级的有效性、高效性和鲁棒性，能够在有限的硬件资源下处理高分辨率的图像。真实的机器人实验表明，该方法同样适用于新型透明物体的鲁棒抓取。完整的数据集和我们的方法可在www.graspnet.net/transcg上公开获得。
Transparent objects are common in our daily life and frequently handled in the automated production line. Robust vision-based robotic grasping and manipulation for these objects would be beneficial for automation. However, the majority of current grasping algorithms would fail in this case since they heavily rely on the depth image, while ordinary depth sensors usually fail to produce accurate depth information for transparent objects owing to the reflection and refraction of light. In this letter, we address this issue by contributing a large-scale real-world dataset for transparent object depth completion, which contains 57,715 RGB-D images from 130 different scenes. Our dataset is the first large-scale, real-world dataset that provides ground truth depth, surface normals, transparent masks in diverse and cluttered scenes. Cross-domain experiments show that our dataset is more general and can enable better generalization ability for models. Moreover, we propose an end-to-end depth completion network, which takes the RGB image and the inaccurate depth map as inputs and outputs a refined depth map. Experiments demonstrate superior efficacy, efficiency and robustness of our method over previous works, and it is able to process images of high resolutions under limited hardware resources. Real robot experiments show that our method can also he applied to novel transparent object grasping robustly. The full dataset and our method are publicly available at www.graspnet.net/transcg.