3D video super-resolution using fully convolutional neural networks
3D video super-resolution using fully convolutional neural networks
复制标题
DOI：
10.1109/icme.2016.7552931
复制
发表时间：
2016-07
期刊：
影响因子：
--
通讯作者：
Yanchun Xie;Jimin Xiao;T. Tillo;Yunchao Wei;Yao Zhao
中科院分区：
文献类型：
--
作者：
Yanchun Xie;Jimin Xiao;T. Tillo;Yunchao Wei;Yao Zhao
大量的冗余信息和庞大的数据量一直是多视点视频系统的严重问题。为了解决这个问题，一个流行的解决方案是混合分辨率，其中只有少数视点保持全分辨率，其他视图保持较低分辨率。在本文中，我们提出了一种超分辨率（SR）方法，其中使用全卷积神经网络对3D视频中的低分辨率视点进行上采样。通过简单地将相邻的高分辨率图像投影到低分辨率图像的位置，我们学习了高分辨率和低分辨率块的关系，并利用投影图像信息将低分辨率图像重建为高分辨率图像。我们建议使用完全卷积神经网络来建立这些图像之间的映射。该网络仅在17对多视图图像上训练，并在其他多视图图像和视频序列上进行测试。据观察，我们提出的方法优于现有的方法客观和主观，超过1 dB的平均增益实现。同时，我们的网络训练过程是高效的，使用一个Titan X GPU不到3小时。
Large amount of redundant information and huge data size have been a serious problem for multiview video systems. To address this problem, one popular solution is mixed-resolution, where only few viewpoints are kept with full resolution and other views are kept with lower resolution. In this paper, we propose a super-resolution (SR) method, where the low-resolution viewpoints in the 3D video are up-sampled using a fully convolutional neural network. By simply projecting the neighboring high resolution image to the position of the low resolution image, we learn the relationship of high and low resolution patches, and reconstruct the low resolution images into high resolution ones using the projected image information. We propose to use a fully convolutional neural network to establish a mapping between those images. The network is barely trained on 17 pairs of multiview images, and tested on other multiview images and video sequences. It is observed that our proposed method outperforms existing methods objectively and subjectively, with more than 1 dB average gain achieved. Meanwhile, our network training procedure is efficient, with less than 3 hours using one Titan X GPU.