Perceptual Deep Depth Super-Resolution
Perceptual Deep Depth Super-Resolution
复制标题
DOI：
10.1109/iccv.2019.00575
复制
发表时间：
2018-12
期刊：
影响因子：
--
通讯作者：
Oleg Voynov;Alexey Artemov;Vage Egiazarian;A. Notchenko;G. Bobrovskikh;Evgeny Burnaev;D. Zorin
中科院分区：
文献类型：
--
作者：
Oleg Voynov;Alexey Artemov;Vage Egiazarian;A. Notchenko;G. Bobrovskikh;Evgeny Burnaev;D. Zorin
RGBD图像结合了来自各种类型深度传感器的高分辨率颜色和低分辨率深度，越来越常见。人们可以通过利用颜色信息来显著提高深度图的分辨率;深度学习方法使颜色和深度信息的结合变得特别容易。然而，融合这两个数据源可能会导致各种工件。如果使用深度图来重建3D形状，例如，对于虚拟现实应用，上采样图像的视觉质量是特别重要的。我们的方法的主要思想是使用生成的3D表面的渲染来测量深度图上采样的质量。我们证明了一个简单的基于视觉外观的损失，当与训练的CNN或简单的深度先验一起使用时，会产生显着改善的3D形状，如通过许多现有的感知指标所测量的。我们比较这种方法与现有的一些优化和学习为基础的技术。
RGBD images, combining high-resolution color and lower-resolution depth from various types of depth sensors, are increasingly common. One can significantly improve the resolution of depth maps by taking advantage of color information; deep learning methods make combining color and depth information particularly easy. However, fusing these two sources of data may lead to a variety of artifacts. If depth maps are used to reconstruct 3D shapes, e.g., for virtual reality applications, the visual quality of upsampled images is particularly important. The main idea of our approach is to measure the quality of depth map upsampling using renderings of resulting 3D surfaces. We demonstrate that a simple visual appearance-based loss, when used with either a trained CNN or simply a deep prior, yields significantly improved 3D shapes, as measured by a number of existing perceptual metrics. We compare this approach with a number of existing optimization and learning-based techniques.