Du2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels
Du2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels
复制标题
DOI：
10.1007/978-3-030-58452-8_34
复制
发表时间：
2020-03
期刊：
影响因子：
--
通讯作者：
Yinda Zhang;N. Wadhwa;Sergio Orts;Christian Häne;S. Fanello;Rahul Garg
中科院分区：
文献类型：
--
作者：
Yinda Zhang;N. Wadhwa;Sergio Orts;Christian Häne;S. Fanello;Rahul Garg
计算立体已经达到了很高的精度水平，但在存在遮挡、重复纹理和沿沿着边缘的对应误差的情况下会降低。我们提出了一种基于神经网络的深度估计新方法，该方法将来自双摄像头的立体声与来自双像素传感器的立体声相结合，这在消费者相机上越来越常见。我们的网络使用了一种新的架构来融合这两个信息源，可以克服纯双目立体匹配的上述局限性。我们的方法提供了一个密集的深度图与尖锐的边缘，这是至关重要的计算摄影应用程序，如合成浅景深或3D照片。此外，我们避免了固有的模糊性，由于立体相机的孔径问题，通过设计的立体基线是正交的双像素基线。我们进行了实验并与最先进的方法进行了比较，以表明我们的方法比以前的工作有了实质性的改进。
Computational stereo has reached a high level of accuracy, but degrades in the presence of occlusions, repeated textures, and correspondence errors along edges. We present a novel approach based on neural networks for depth estimation that combines stereo from dual cameras with stereo from a dual-pixel sensor, which is increasingly common on consumer cameras. Our network uses a novel architecture to fuse these two sources of information and can overcome the above-mentioned limitations of pure binocular stereo matching. Our method provides a dense depth map with sharp edges, which is crucial for computational photography applications like synthetic shallow-depth-of-field or 3D Photos. Additionally, we avoid the inherent ambiguity due to the aperture problem in stereo cameras by designing the stereo baseline to be orthogonal to the dual-pixel baseline. We present experiments and comparisons with state-of-the-art approaches to show that our method offers a substantial improvement over previous works.