Single-Image Piece-Wise Planar 3D Reconstruction via Associative Embedding
Single-Image Piece-Wise Planar 3D Reconstruction via Associative Embedding
复制标题
DOI：
10.1109/cvpr.2019.00112
复制
发表时间：
2019-02
期刊：
影响因子：
--
通讯作者：
Zehao Yu;Jia Zheng;Dongze Lian;Zihan Zhou;Shenghua Gao
中科院分区：
文献类型：
--
作者：
Zehao Yu;Jia Zheng;Dongze Lian;Zihan Zhou;Shenghua Gao
单位图片平面3D重建的目的是同时段平面实例并从图像中恢复3D平面参数。最近的方法利用卷积神经网络（CNN）并取得了令人鼓舞的结果。但是，这些方法仅限于检测具有某些学习顺序的固定平面。为了解决这个问题，我们提出了一种基于关联嵌入的新型两阶段方法，其灵感来自于其在实例细分中的最新成功。在第一阶段，我们训练一个CNN将每个像素映射到一个嵌入空间，在该空间中，来自同一平面实例的像素具有相似的嵌入。然后，通过通过有效的平均偏移聚类算法将嵌入向量分组来获得平面实例。在第二阶段，我们通过考虑像素级和实例级的一致性来估计每个平面实例的参数。通过提出的方法，我们能够检测任意数量的平面。公共数据集的广泛实验验证了我们方法的有效性和效率。此外，我们的方法在测试时间以30 fps的速度运行，因此可以促进许多实时应用，例如视觉大满贯和人类机器人相互作用。代码可从https://github.com/svip-lab/planarreconstruction获得。
Single-image piece-wise planar 3D reconstruction aims to simultaneously segment plane instances and recover 3D plane parameters from an image. Most recent approaches leverage convolutional neural networks (CNNs) and achieve promising results. However, these methods are limited to detecting a fixed number of planes with certain learned order. To tackle this problem, we propose a novel two-stage method based on associative embedding, inspired by its recent success in instance segmentation. In the first stage, we train a CNN to map each pixel to an embedding space where pixels from the same plane instance have similar embeddings. Then, the plane instances are obtained by grouping the embedding vectors in planar regions via an efficient mean shift clustering algorithm. In the second stage, we estimate the parameter for each plane instance by considering both pixel-level and instance-level consistencies. With the proposed method, we are able to detect an arbitrary number of planes. Extensive experiments on public datasets validate the effectiveness and efficiency of our method. Furthermore, our method runs at 30 fps at the testing time, thus could facilitate many real-time applications such as visual SLAM and human-robot interaction. Code is available at https://github.com/svip-lab/PlanarReconstruction.