PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo
PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo
复制标题
DOI：
10.1109/cvpr52688.2022.00847
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Jiacheng Liu;Pan Ji;Nitin Bansal;Changjiang Cai;Qingan Yan;Xiaolei Huang;Yi Xu
中科院分区：
文献类型：
--
作者：
Jiacheng Liu;Pan Ji;Nitin Bansal;Changjiang Cai;Qingan Yan;Xiaolei Huang;Yi Xu
我们从具有已知相机姿势的多个输入视图中提出了一个名为3D平面重建的新型框架。大多数以前基于学习的平面重建方法从单个图像中重建了3D平面，这些图像高度依赖于单视回归并受到深度尺度歧义的困扰。相比之下，我们使用利用多视图几何形状的多视图（MVS）管道重建3D平面。我们将平面重建为语义平面检测分支和平面MVS分支。语义平面检测分支基于单视平面检测框架，但有差异。平面MVS分支采用一组倾斜的平面假设来替换传统的深度假设以执行平面扫描策略，并最终学习像素级平面参数及其平面深度图。我们介绍了如何以平衡的方式学习两个分支，并提出软性损失，以使两个分支的输出相关联并使其彼此受益。在各种室内数据集上进行的广泛实验表明，在平面检测和3D几何指标上，PlaneMV显着超过了最先进（SOTA）的单视平面重建方法。由于学识渊博的平面先验，我们的方法甚至优于基于SOTA学习的MVS方法。据我们所知，这是端到端MVS框架内3D平面重建的第一部作品。
We present a novel framework named PlaneMVS for 3D plane reconstruction from multiple input views with known camera poses. Most previous learning-based plane reconstruction methods reconstruct 3D planes from single images, which highly rely on single-view regression and suffer from depth scale ambiguity. In contrast, we reconstruct 3D planes with a multi-view-stereo (MVS) pipeline that takes advantage of multi-view geometry. We decouple plane reconstruction into a semantic plane detection branch and a plane MVS branch. The semantic plane detection branch is based on a single-view plane detection framework but with differences. The plane MVS branch adopts a set of slanted plane hypotheses to replace conventional depth hypotheses to perform plane sweeping strategy and finally learns pixel-level plane parameters and its planar depth map. We present how the two branches are learned in a balanced way, and propose a soft-pooling loss to associate the outputs of the two branches and make them benefit from each other. Extensive experiments on various indoor datasets show that PlaneMVS significantly outperforms state-of-the-art (SOTA) single-view plane reconstruction methods on both plane detection and 3D geometry metrics. Our method even outperforms a set of SOTA learning-based MVS methods thanks to the learned plane priors. To the best of our knowledge, this is the first work on 3D plane reconstruction within an end-to-end MVS framework.