Applying VertexShuffle toward 360-degree video super-resolution
Applying VertexShuffle toward 360-degree video super-resolution
复制标题
DOI：
10.1145/3534088.3534353
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
N. Li;Yao Liu
中科院分区：
文献类型：
--
作者：
N. Li;Yao Liu
随着深度学习模型最近的成功，2D图像超分辨率的性能得到了显着提高。受最近最先进的2D超分辨率模型和球形CNN的启发，本文设计了一种用于360度视频的新型球形超分辨率（SSR）方法。为了解决与360度视频传输/流相关的带宽浪费问题并节省计算，我们提出了聚焦二十面体网格来表示球体上的一个小区域，并构造矩阵来将球形内容旋转到聚焦网格区域。我们还提出了一种新的顶点洗牌操作的网格，由2D PixelShuffle操作的动机。我们将我们的SSR方法与最先进的2D超分辨率模型进行了比较。我们表明，SSR有可能实现显着的好处时，适用于球形信号。
With the recent successes of deep learning models, the performance of 2D image super-resolution has improved significantly. Inspired by recent state-of-the-art 2D super-resolution models and spherical CNNs, in this paper, we design a novel spherical superresolution (SSR) approach for 360-degree videos. To address the bandwidth waste problem associated with 360-degree video transmission/streaming and save computation, we propose the Focused Icosahedral Mesh to represent a small area on the sphere and construct matrices to rotate spherical content to the focused mesh area. We also propose a novel VertexShuffle operation on the mesh, motivated by the 2D PixelShuffle operation. We compare our SSR approach with state-of-the-art 2D super-resolution models. We show that SSR has the potential to achieve significant benefits when applied to spherical signals.