Generative Video Compression with a Transformer-Based Discriminator
Generative Video Compression with a Transformer-Based Discriminator
复制标题
DOI：
10.1109/pcs56426.2022.10018030
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Pengli Du;Y. Liu;Nam Ling;Yongxiong Ren;Lingzhi Liu
中科院分区：
文献类型：
--
作者：
Pengli Du;Y. Liu;Nam Ling;Yongxiong Ren;Lingzhi Liu
深度学习已成功应用于图像和视频压缩。具体而言，生成的对抗网络（GAN）可以以较低的细节和高感知质量以低比特速率压缩图像。在这项工作中，我们提出了一种具有基于变压器的歧视器（TD）的新颖生成视频压缩（GVC）模型，该模型在视频框架内学习非本地相关性以改善对抗性训练。此外，我们的GVC模型还结合了训练发电机的新损失，该损失结合了基本损失，依赖歧视者的特征损失和感知损失。 HEVC测试序列的实验表明，与现有的学识渊博和传统视频编码方案相比，提出的GVC模型以极低的比特率提供了出色的性能。
Deep learning has been successfully applied to image and video compression. Specifically, generative adversarial network (GAN) can compress images at low bit rates with sharp details and high perceptual quality. In this work, we propose a novel generative video compression (GVC) model with a transformer-based discriminator (TD), which learns non-local correlations within video frames to improve adversarial training. Besides, our GVC model incorporates a new loss to train the generator, which combines a base loss, a discriminator-dependent feature loss, and a perceptual loss. Experiments on HEVC test sequences demonstrate that the proposed GVC model provides superior performance at extremely low bit rates, compared to existing learned and traditional video coding schemes.