Learning Category-Level Manipulation Tasks from Point Clouds with Dynamic Graph CNNs
Learning Category-Level Manipulation Tasks from Point Clouds with Dynamic Graph CNNs
复制标题
DOI：
10.1109/icra48891.2023.10160820
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Junchi Liang;Abdeslam Boularias
中科院分区：
文献类型：
--
作者：
Junchi Liang;Abdeslam Boularias
本文提出了一种从任务演示的原始 RGB-D 视频中学习类别级操作的新技术，无需手动标签或注释。类别级学习旨在获得可推广到新对象的技能，这些新对象的几何形状和纹理与演示中使用的对象不同。我们通过首先将抓取和操作视为工具使用的特殊情况来解决这个问题，其中工具对象被移动到目标对象的参考系中定义的一系列关键姿势。使用动态图卷积神经网络来预测工具和目标对象及其关键姿势，该网络将整个场景的自动分段深度和彩色图像作为输入。使用真实机械臂进行对象操作任务的经验结果表明，所提出的网络可以有效地从真实的视觉演示中学习，以在同一类别内的新对象上执行任务，并且优于替代方法。
This paper presents a new technique for learning category-level manipulation from raw RGB-D videos of task demonstrations, with no manual labels or annotations. Category-level learning aims to acquire skills that can be generalized to new objects, with geometries and textures that are different from the ones of the objects used in the demonstrations. We address this problem by first viewing both grasping and manipulation as special cases of tool use, where a tool object is moved to a sequence of key-poses defined in a frame of reference of a target object. Tool and target objects, along with their key-poses, are predicted using a dynamic graph convolutional neural network that takes as input an automatically segmented depth and color image of the entire scene. Empirical results on object manipulation tasks with a real robotic arm show that the proposed network can efficiently learn from real visual demonstrations to perform the tasks on novel objects within the same category, and outperforms alternative approaches.