Fast Joint Estimation of Silhouettes and Dense 3D Geometry from Multiple Images
Fast Joint Estimation of Silhouettes and Dense 3D Geometry from Multiple Images
复制标题
DOI：
10.1109/tpami.2011.150
复制
发表时间：
2012-03-01
影响因子：
23.6
通讯作者：
Cremers, Daniel
中科院分区：
文献类型：
--
作者：
Kolev, Kalin;Brox, Thomas;Cremers, Daniel
我们提出了一个概率公式的关节轮廓提取和三维重建给出了一系列校准的二维图像。我们不是单独分割每个图像以构建与估计轮廓一致的3D表面，而是计算最可能的3D形状，从而产生观察到的颜色信息。基于贝叶斯推理的概率框架，通过最佳地考虑所有视图的贡献，实现鲁棒的3D重建。我们利用凸松弛技术在空间连续表示中以全局最优的方式解决了产生的最大后验形状推理。对于以涂鸦形式指定前景和背景区域的交互式提供的用户输入，我们将相应的颜色分布构建为多变量高斯分布，并在变分意义上找到最适合该数据的体积占用。与基于轮廓的经典多视图重建方法相比，该方法不依赖于初始化，并且对背景杂波、镜面反射和相机传感器摄动导致的模型假设违反具有显著的弹性。在几个真实世界数据集的实验中，我们表明，在多视图设置中利用轮廓一致性标准可以显著改善独立2D分割的轮廓质量，而不会显著增加计算工作量。这导致更准确的视觉船体估计，需要大量的基于图像的建模方法。我们利用并行计算的最新进展，用GPU实现所提出的方法，在4.41秒内在超过2000万体素的体网格上生成重建。
We propose a probabilistic formulation of joint silhouette extraction and 3D reconstruction given a series of calibrated 2D images. Instead of segmenting each image separately in order to construct a 3D surface consistent with the estimated silhouettes, we compute the most probable 3D shape that gives rise to the observed color information. The probabilistic framework, based on Bayesian inference, enables robust 3D reconstruction by optimally taking into account the contribution of all views. We solve the arising maximum a posteriori shape inference in a globally optimal manner by convex relaxation techniques in a spatially continuous representation. For an interactively provided user input in the form of scribbles specifying foreground and background regions, we build corresponding color distributions as multivariate Gaussians and find a volume occupancy that best fits to this data in a variational sense. Compared to classical methods for silhouette-based multiview reconstruction, the proposed approach does not depend on initialization and enjoys significant resilience to violations of the model assumptions due to background clutter, specular reflections, and camera sensor perturbations. In experiments on several real-world data sets, we show that exploiting a silhouette coherency criterion in a multiview setting allows for dramatic improvements of silhouette quality over independent 2D segmentations without any significant increase of computational efforts. This results in more accurate visual hull estimation, needed by a multitude of image-based modeling approaches. We made use of recent advances in parallel computing with a GPU implementation of the proposed method generating reconstructions on volume grids of more than 20 million voxels in up to 4.41 seconds.