Structure from Action: Learning Interactions for 3D Articulated Object Structure Discovery
Structure from Action: Learning Interactions for 3D Articulated Object Structure Discovery
复制标题
DOI：
10.1109/iros55552.2023.10342135
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Neil Nie;S. Gadre;Kiana Ehsani;Shuran Song
中科院分区：
文献类型：
--
作者：
Neil Nie;S. Gadre;Kiana Ehsani;Shuran Song
我们介绍结构从行动（SfA），一个框架，发现3D部分的几何形状和关节参数看不见的关节连接的对象通过一系列的推断的相互作用。我们的关键见解是，应结合考虑3D交互和感知来构建3D铰接式CAD模型，特别是对于培训期间未看到的类别。通过选择信息交互，Sf A发现零件并显示遮挡表面，就像关闭的抽屉内部一样。通过在3D中聚合视觉观察，Sf A准确地分割多个零件，重建零件几何形状，并在规范坐标系中推断所有关节参数。我们的实验表明，在模拟训练的Sf A模型可以推广到许多看不见的对象类别与不同的结构和现实世界的对象。从经验上看，Sf A在不可见类别上的性能比最先进组件管道高出25.4个3D IoU百分点，同时与已经具有性能的联合估计基准相匹配。11有关代码、数据和视频，请参阅sfa.cs.columbia.edu/
We introduce Structure from Action (SfA), a framework to discover 3D part geometry and joint parameters of unseen articulated objects via a sequence of inferred interactions. Our key insight is that 3D interaction and perception should be considered in conjunction to construct 3D articulated CAD models, especially for categories not seen during training. By selecting informative interactions, Sf A discovers parts and reveals occluded surfaces, like the inside of a closed drawer. By aggregating visual observations in 3D, Sf A accurately segments multiple parts, reconstructs part geometry, and infers all joint parameters in a canonical coordinate frame. Our experiments demonstrate that a Sf A model trained in simulation can generalize to many unseen object categories with diverse structures and to real-world objects. Empirically, Sf A outperforms a pipeline of state-of-the-art components by 25.4 3D IoU percentage points on unseen categories, while matching already performant joint estimation baselines.11For code, data, and videos, see sfa.cs.columbia.edu/