Learning 3D Part Assembly from a Single Image
Learning 3D Part Assembly from a Single Image
复制标题
DOI：
10.1007/978-3-030-58539-6_40
复制
发表时间：
2020-03
期刊：
影响因子：
--
通讯作者：
Yichen Li;Kaichun Mo;Lin Shao;Minhyuk Sung;L. Guibas
中科院分区：
文献类型：
--
作者：
Yichen Li;Kaichun Mo;Lin Shao;Minhyuk Sung;L. Guibas
在许多应用中，自主装配是机器人的一项重要能力。在这项任务中，避障、运动规划和执行器控制等问题在机器人领域得到了广泛的研究。然而，当谈到任务规范时，可能性的空间仍然没有得到充分的探索。为此，我们引入了一个新的问题，即单图像引导的3D零件装配，以及一个基于学习的解决方案。我们从给定的一组完整的零件和描述整个装配对象的单一图像出发，在家具装配的设置中研究这个问题。在这种设置中存在多个挑战，包括处理部件之间的歧义(例如，椅子靠背和腿部担架中的板条)以及部件和零件子组件的3D姿势预测，无论是可见的还是遮挡的。我们提出了一种两模块流水线来解决这些问题，该流水线利用强大的2D-3D对应关系和面向装配的图形消息传递来推断零件关系。在基于PartNet的合成基准测试的实验中，我们与三种基准方法(代码和数据可在https://github.com/AntheaLi/3DPartAssembly上获得)进行了比较，证明了该框架的有效性。
Autonomous assembly is a crucial capability for robots in many applications. For this task, several problems such as obstacle avoidance, motion planning, and actuator control have been extensively studied in robotics. However, when it comes to task specification, the space of possibilities remains underexplored. Towards this end, we introduce a novel problem,single-image-guided 3D part assembly, along with a learning-based solution. We study this problem in the setting offurniture assemblyfrom a given complete set of parts and a single image depicting the entire assembled object. Multiple challenges exist in this setting, including handling ambiguity among parts (e.g., slats in a chair back and leg stretchers) and 3D pose prediction for parts and part subassemblies, whether visible or occluded. We address these issues by proposing a two-module pipeline that leverages strong 2D-3D correspondences and assembly-oriented graph message-passing to infer part relationships. In experiments with a PartNet-based synthetic benchmark, we demonstrate the effectiveness of our framework as compared with three baseline approaches (code and data available at https://github.com/AntheaLi/3DPartAssembly ).