360MVSNet: Deep Multi-view Stereo Network with 360° Images for Indoor Scene Reconstruction
360MVSNet: Deep Multi-view Stereo Network with 360° Images for Indoor Scene Reconstruction
复制标题
DOI：
10.1109/wacv56688.2023.00307
复制
发表时间：
2023-01
期刊：
影响因子：
--
通讯作者：
Ching-Ya Chiu;Yu-Ting Wu;I-Chao Shen;Yung-Yu Chuang
中科院分区：
文献类型：
--
作者：
Ching-Ya Chiu;Yu-Ting Wu;I-Chao Shen;Yung-Yu Chuang
随着深度学习技术的进步，近年来的多视点立体视觉方法取得了可喜的成果。尽管取得了进步，但由于常规图像的视野有限，重建大型室内环境仍然需要收集许多视觉重叠足够的图像，这是相当劳动密集型的。360°图像覆盖的视野比常规图像大得多，并且有助于捕获过程。在本文中，我们提出了360MVSNet，这是第一个用于360°图像的多视图立体深度学习网络。我们的方法将不确定性估计与从多个视点捕获的360°图像的球面扫描模块相结合，以构建多尺度成本体。通过对体进行从粗到精的回归，可以得到高分辨率的深度图。此外，我们还构建了EQMVS，这是一个由超过50K对等矩形投影的RGB和深度图组成的大规模合成数据集。实验结果表明，该方法可以重建大型合成和真实室内场景，其完整性明显优于传统方法和基于学习的方法，同时节省了数据采集过程中的时间和精力。
Recent multi-view stereo methods have achieved promising results with the advancement of deep learning techniques. Despite of the progress, due to the limited fields of view of regular images, reconstructing large indoor environments still requires collecting many images with sufficient visual overlap, which is quite labor-intensive. 360° images cover a much larger field of view than regular images and would facilitate the capture process. In this paper, we present 360MVSNet, the first deep learning network for multi-view stereo with 360° images. Our method combines uncertainty estimation with a spherical sweeping module for 360° images captured from multiple viewpoints in order to construct multi-scale cost volumes. By regressing volumes in a coarse-to-fine manner, high-resolution depth maps can be obtained. Furthermore, we have constructed EQMVS, a large-scale synthetic dataset that consists of over 50K pairs of RGB and depth maps in equirectangular projection. Experimental results demonstrate that our method can reconstruct large synthetic and real-world indoor scenes with significantly better completeness than previous traditional and learning-based methods while saving both time and effort in the data acquisition process.