Depth map upsampling using joint edge-guided convolutional neural network for virtual view synthesizing
Depth map upsampling using joint edge-guided convolutional neural network for virtual view synthesizing
复制标题
使用联合边缘引导卷积神经网络进行深度图上采样以进行虚拟视图合成
DOI：
10.1117/1.jei.26.4.043004
复制
发表时间：
2017-07
影响因子：
1.1
通讯作者：
Chao Yao
中科院分区：
文献类型：
--
作者：
Yan Dong;Chunyu Lin;Yao Zhao;Chao Yao
在纹理加深度的三维视觉数据格式中，纹理图和深度图都需要通过基于深度图像的渲染来合成所需的视图。然而，与相应的纹理图像相比，捕获或估计的深度图总是存在低分辨率。提出了一种联合边缘引导卷积神经网络，在保证合成视点质量的前提下提高深度图的分辨率。该网络以深度图提取的联合边缘特征和配准的纹理图像作为参考，将低分辨率深度图作为输入，生成高分辨率深度图。我们进一步使用局部约束来保持平滑区域和锐利边缘，从而提高深度图和合成视图的质量。最后，在恢复过程中以虚拟视图质量为指导进行全局循环优化。我们使用深度图和纹理图像对我们的模型进行训练，然后在其他深度图和视频序列上进行测试。实验结果表明，我们的方案在深度图和合成视图的质量上都优于现有的方法。
In texture-plus-depth format of three-dimensional visual data, both texture and depth maps are required to synthesize a desired view via depth-image-based rendering. However, the depth maps captured or estimated always exist with low resolution compared to their corresponding texture images. We introduce a joint edge-guided convolutional neural network that upsamples the resolution of a depth map on the premise of synthesized view quality. The network takes the low-resolution depth map as an input using a joint edge feature extracted from the depth map and the registered texture image as a reference, and then produces a high-resolution depth map. We further use local constraints that preserve smooth regions and sharp edges so as to improve the quality of the depth map and synthesized view. Finally, a global looping optimization is performed with virtual view quality as guidance in the recovery process. We train our model using pairs of depth maps and texture images and then make tests on other depth maps and video sequences. The experimental results demonstrate that our scheme outperforms existing methods both in the quality of the depth maps and synthesized views.
登录
查看更多内容
DOI：
10.1109/icassp.2016.7472127
发表时间：
2016-03
期刊：
2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
影响因子：
--
作者：
Xin Zhang;Ruiyuan Wu
通讯作者：
Xin Zhang;Ruiyuan Wu
影响因子：
10.6
作者：
Yang, Jianchao;Wright, John;Ma, Yi
通讯作者：
Ma, Yi
DOI：
10.1007/978-3-642-10467-1_119
发表时间：
2009-12
期刊：
--
影响因子：
--
作者：
Jian Zhang;Siwei Ma;Yongbing Zhang;Debin Zhao
通讯作者：
Jian Zhang;Siwei Ma;Yongbing Zhang;Debin Zhao
DOI：
10.1109/icme.2016.7552931
发表时间：
2016-07
期刊：
2016 IEEE International Conference on Multimedia and Expo (ICME)
影响因子：
--
作者：
Yanchun Xie;Jimin Xiao;T. Tillo;Yunchao Wei;Yao Zhao
通讯作者：
Yanchun Xie;Jimin Xiao;T. Tillo;Yunchao Wei;Yao Zhao
影响因子：
10.6
作者：
Sheikh, HR;Bovik, AC;de Veciana, G
通讯作者：
de Veciana, G