Video Super-Resolution With Convolutional Neural Networks
Video Super-Resolution With Convolutional Neural Networks
复制标题
DOI：
10.1109/tci.2016.2532323
复制
发表时间：
2016-06-01
影响因子：
5.4
通讯作者：
Katsaggelos, Aggelos K.
中科院分区：
文献类型：
--
作者：
Kappeler, Armin;Yoo, Seunghwan;Katsaggelos, Aggelos K.
卷积神经网络(CNN)是深度神经网络(DNN)的一种特殊类型。到目前为止，它们已经成功地应用于图像超分辨率(SR)以及其他图像恢复任务。在本文中，我们考虑了视频超分辨率问题。我们提出了一种同时在视频的空间和时间维度上进行训练的CNN，以提高视频的空间分辨率。连续的帧经过运动补偿，用作CNN的输入，CNN提供超分辨率视频帧作为输出。我们研究了在一个CNN架构中组合视频帧的不同选项。虽然有大量的图像数据库可用于训练深度神经网络，但创建一个足够高质量的大型视频数据库来训练用于视频恢复的神经网络是更具挑战性的。我们表明，通过使用图像来预训练我们的模型，相对较小的视频数据库足以训练我们的模型来实现甚至改进当前的最先进水平。我们将我们提出的方法与当前的视频以及图像SR算法进行了比较。
Convolutional neural networks (CNN) are a special type of deep neural networks (DNN). They have so far been successfully applied to image super-resolution (SR) as well as other image restoration tasks. In this paper, we consider the problem of video super-resolution. We propose a CNN that is trained on both the spatial and the temporal dimensions of videos to enhance their spatial resolution. Consecutive frames are motion compensated and used as input to a CNN that provides super-resolved video frames as output. We investigate different options of combining the video frames within one CNN architecture. While large image databases are available to train deep neural networks, it is more challenging to create a large video database of sufficient quality to train neural nets for video restoration. We show that by using images to pretrain our model, a relatively small video database is sufficient for the training of our model to achieve and even improve upon the current state-of-the-art. We compare our proposed approach to current video as well as image SR algorithms.