MVS2D: Efficient Multiview Stereo via Attention-Driven 2D Convolutions
MVS2D: Efficient Multiview Stereo via Attention-Driven 2D Convolutions
复制标题
DOI：
10.1109/cvpr52688.2022.00838
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Zhenpei Yang;Zhile Ren;Qi Shan;Qi-Xing Huang
中科院分区：
文献类型：
--
作者：
Zhenpei Yang;Zhile Ren;Qi Shan;Qi-Xing Huang
深度学习对多视图立体系统产生了重大影响。最先进的方法通常涉及构建成本量，然后进行多个 3D 卷积运算以恢复输入图像的像素深度。虽然这种平面扫描立体声的端到端学习提高了公共基准的准确性，但它们的计算速度通常非常慢。我们提出了 MVS2D，一种高效的多视图立体算法，它通过注意力机制将多视图约束无缝集成到单视图网络中。由于 MVS2D 仅基于 2D 卷积构建，因此它比所有著名的同类产品至少快 2 倍。此外，我们的算法可生成精确的深度估计和 3D 重建，在具有挑战性的基准 ScanNet、SUN3D、RGBD 和经典 DTU 数据集上取得最先进的结果。在不精确的相机姿势设置中，我们的算法也优于所有其他算法。我们的代码发布在https://github.com/zhenpeiyang/MVS2D
Deep learning has made significant impacts on multiview stereo systems. State-of-the-art approaches typically involve building a cost volume, followed by multiple 3D convolution operations to recover the input image's pixel-wise depth. While such end-to-end learning of plane-sweeping stereo advances public benchmarks' accuracy, they are typically very slow to compute. We present MVS2D, a highly efficient multi-view stereo algorithm that seamlessly integrates multi-view constraints into single-view net-works via an attention mechanism. Since MVS2D only builds on 2D convolutions, it is at least $2\times faster$ than all the notable counterparts. Moreover, our algorithm produces precise depth estimations and 3D reconstructions, achieving state-of-the-art results on challenging benchmarks ScanNet, SUN3D, RGBD, and the classical DTU dataset. our algorithm also outperforms all other algorithms in the setting of inexact camera poses. Our code is released at https://github.com/zhenpeiyang/MVS2D