Enhanced Surveillance Video Compression With Dual Reference Frames Generation
Enhanced Surveillance Video Compression With Dual Reference Frames Generation
复制标题
通过双参考帧生成增强监控视频压缩
DOI：
10.1109/tcsvt.2021.3073114
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Wen Gao
中科院分区：
文献类型：
--
作者：
Lei Zhao;Shiqi Wang;Shanshe Wang;Yan Ye;Siwei Ma;Wen Gao
本文通过同时研究背景和前景冗余的不同特性来提高监控视频的帧间编码性能，并以互补的方式引入了两个新的参考帧。一方面，为了减少背景冗余，提出了块级背景参考帧(BRF)。该方案将语义信息融入到压缩过程中，并利用实例分割来辅助背景块决策，使生成的BRF不受前景污染。另一方面，为了处理前景冗余，基于监视预测生成对抗网络(SP-GAN)生成前景参考帧(FRF)，其利用先前重构的帧、基于光流的预测以及BRF来推断待编码帧的前景对象。我们将该方案集成到HM-16.6软件中，并将BRF和FRF添加到参考图片列表(RPS)中。仿真结果表明，该方案具有较大的优越性。特别是，通过将所提出的BRF添加到RPS，与现有的BRF方法相比，观察到3%的编码增益。当将BRF和FRF合并到RPS中时，监控视频编码可以获得5.8%的增益。
In this paper, we improve the inter coding performance of surveillance videos by simultaneously investigating the distinct characteristics of background and foreground redundancy, and introduce two novel reference frames in a complementary manner. On one hand, a block level background reference frame (BRF) is proposed to reduce the background redundancy. The proposed scheme incorporates semantic information into the compression process, and makes use of instance segmentation to facilitate the background block decision, making the generated BRF free from foreground pollution. On the other hand, in order to handle foreground redundancy, a foreground reference frame (FRF) is generated based on Surveillance Prediction Generative Adversarial Network (SP-GAN), which utilizes previous reconstructed frames, optical flow based prediction, as well as BRF to infer the foreground objects of the to-be-coded frame. We integrate the proposed scheme into HM-16.6 software and append BRF and FRF to the reference pictures list (RPS). Simulation results demonstrate considerable superiority of the proposed scheme. In particular, by adding the proposed BRF to RPS, 3% coding gains are observed compared with the state-of-the-art BRF method. When both BRF and FRF are incorporated into RPS, 5.8% gains are achieved for surveillance video coding.