Spatio-Temporal Convolutional Neural Network for Frame Rate Up-Conversion
Spatio-Temporal Convolutional Neural Network for Frame Rate Up-Conversion
复制标题
DOI：
10.1145/3325773.3325777
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Yusuke Tanaka;T. Omori
中科院分区：
文献类型：
--
作者：
Yusuke Tanaka;T. Omori
通过实现更高的分辨率和更高的帧速率来提高视频的视觉质量。为了实现更高的帧速率，我们提出了一种新的帧速率上转换方法使用时空卷积神经网络。近年来，随着卷积神经网络等机器学习技术的发展，实现了更清晰的插值帧估计。然而，利用传统的卷积神经网络方法，很难为包括复杂运动的视频估计准确的内插帧。为了解决这个问题，我们采用了时空卷积而不是传统的空间卷积。时空卷积被认为是有效的非线性运动，因为它可以捕捉到的对象的运动的时间变化。我们通过使用包含旋转运动和缩放等复杂运动的视频数据验证了所提出方法的有效性。
The visual quality of the video is improved by realizing higher resolution and higher frame rate. In order to realize higher frame rate, we propose new frame rate up-conversion method using spatio-temporal convolutional neural network. In recent years, with the development of machine learning techniques such as convolutional neural networks, clearer interpolation frame estimation has been realized. However, with the conventional convolutional neural network method, it is difficult to estimate an accurate interpolation frames for video including complex motion. In order to deal with this problem, we adopted spatio-temporal convolution rather than conventional spatial convolution. Spatio-temporal convolution is thought to be effective for nonlinear motion because it can capture the time change of the motion of the object. We verified the effectiveness of the proposed method by using video data including complex motions such as rotational motion and scaling.