Automatic Mechanism Modeling from a Single Image with CNNs
Automatic Mechanism Modeling from a Single Image with CNNs
复制标题
使用 CNN 从单个图像进行自动机构建模
DOI：
10.1111/cgf.13572
复制
发表时间：
2018
影响因子：
2.5
通讯作者：
Yang Yin
中科院分区：
文献类型：
--
作者：
Lin Minmin;Shao Tianjia;Zheng Youyi;Ren Zhong;Weng Yanlin;Yang Yin
本文提出了一种新颖的系统，该系统可以从单个RGB图像中对机构组件的三维几何形状和功能进行全自动建模。最终的3D机构模型与输入图像中的模型高度相似，具有物理有效方式规定的所有机械部件的几何形状，机械属性，连接性和功能。这个具有挑战性的任务是通过结合各种深度卷积神经网络来实现的，为每个单独的零件组件提供高质量和自动的零件检测、分割、相机姿态估计和机械属性检索。在此基础上，我们使用局部/全局优化算法来建立所有部件之间的几何相互依赖关系，同时保持其期望的空间排列。我们使用交互图来抽象所得到的机构系统的部件间连接。如果在图中识别出一个孤立的组件，我们的系统列举所有可能的解决方案来恢复图的连通性，并输出残差最小的解决方案。我们已经用大量的经典机构照片对我们的系统进行了广泛的测试，实验结果表明，所提出的系统能够在没有用户指导的情况下构建高质量的3D机构模型。
This paper presents a novel system that enables a fully automatic modeling of both 3D geometry and functionality of a mechanism assembly from a single RGB image. The resulting 3D mechanism model highly resembles the one in the input image with the geometry, mechanical attributes, connectivity, and functionality of all the mechanical parts prescribed in a physically valid way. This challenging task is realized by combining various deep convolutional neural networks to provide high‐quality and automatic part detection, segmentation, camera pose estimation and mechanical attributes retrieval for each individual part component. On the top of this, we use a local/global optimization algorithm to establish geometric interdependencies among all the parts while retaining their desired spatial arrangement. We use an interaction graph to abstract the inter‐part connection in the resulting mechanism system. If an isolated component is identified in the graph, our system enumerates all the possible solutions to restore the graph connectivity, and outputs the one with the smallest residual error. We have extensively tested our system with a wide range of classic mechanism photos, and experimental results show that the proposed system is able to build high‐quality 3D mechanism models without user guidance.