Conditional Coding for Learned Image and Video Compression
Conditional Coding for Learned Image and Video Compression
批准号:
508272532
负责人:
Professor Dr.-Ing. Jörn Ostermann
金额:
$0.0万
依托单位国家:
德国
项目类别:
Research Grants
财政年份:
--
资助国家:
德国
项目状态:
未结题
起止时间:
中文摘要
深度学习的到来激发了端到端学习压缩的新浪潮。近年来,学习图像压缩取得了成功,最先进的MS-SSIM结果优于VVC Intra (PSNR结果与VVC Intra相当)。相比之下,端到端学习视频压缩的发展还处于早期阶段。大多数学习的视频编解码器遵循传统的基于混合的编码架构,即时间预测,然后是基于变换的残差编码。最近的一份出版物表明,尽管最先进的学习视频编解码器显示出比x265更好的结果,但在更现实的测试条件下,它们很难与HEVC测试模型(HM)竞争。最近,出现了一种新的思想流派,称为帧间条件编码,将端到端学习视频编码提升到一个新的压缩性能水平。条件编码的思想是学习以有用的上下文信息为条件的编码帧的数据分布,以便达到较低的条件熵率以获得更好的压缩。深度生成模型的出现,如变分自编码器(VAE)和归一化流模型,为基于学习的压缩的范式转变开辟了新的机会。目前,VAE是压缩主干的常用选择。作为一种新的尝试,本联合研究方案引入了一种特殊类型的归一化流模型,称为增强归一化流(ANF),用于条件编码。我们选择ANF是因为它比VAE具有更好的表达能力,并且将VAE作为一个特例。这个联合研究项目的另一个值得注意的方面是解决学习视频编解码器的通用性和适应性。学习到的编解码器往往存在训练数据和测试数据之间的域差距;也就是说,它们可能不能很好地概括看不见的数据。从更一般的意义上说,它们很难实现对单个测试图像/视频的最佳压缩,每个测试图像/视频实际上都可以被认为是一个不同的域。为了提高可泛化性,本提案应以元学习的形式纳入诺特定理,学习归纳偏差,促使解码后的视频帧在时间维度上保持一定的潜在一致性。我们还将使用这种习得的归纳偏置来在推理时调整编码器和/或解码器,以适应个别视频。由于其无监督的性质,我们的方法具有不必在比特流中发出任何附加信息的显著特征。
英文摘要
This joint research project between the Institut für Informationsverarbeitung (TNT) of the Leibniz Universität Hannover (LUH) and the Department of Computer Science of the National Chiao Tung University (NYCU) in Taiwan addresses end-to-end learned video compression from the perspective of conditional coding with an meta learning-based regularization and tailoring scheme.The arrival of deep learning spurs a new wave of developments in end-to-end learned compression. Recent years witnessed the success of learned image compression, with the state-of-the-art showing better MS-SSIM results than (and comparable PSNR results to) VVC Intra. By comparison, the development of end-to-end learned video compression is still in its early stage. Most learned video codecs follow the traditional, hybrid-based coding architecture, namely temporal prediction followed by transform-based residual coding. A recent publication indicates that although the state-of-the-art learned video codecs show better results than x265, they can hardly compete with the HEVC Test Model (HM) under more realistic test conditions.Recently, a new school of thought, known as inter-frame conditional coding, emerged, taking end-to-end learned video coding to a new level of compression performance. The idea of conditional coding is to learn the data distribution of a coding frame conditioned on useful contextual information, in order to reach a lower conditional entropy rate for better compression.The emergence of deep generative models, such as variational autoencoders (VAE) and normalizing flow models, opens up new opportunities for a paradigm shift in learning-based compression. Currently, VAE is a popular choice for the compression backbone. Representing a new attempt, this joint research proposal introduces a special type of normalizing flow model, called augmented normalizing flows (ANF), for conditional coding. We choose ANF because it is shown to achieve superior expressiveness to VAE and includes VAE as a special case.Another notable aspect of this joint research project is to address the generalizability and adaptability of the learned video codecs. The learned codecs often suffer from the domain gap between the training and the test data; that is, they may not generalize well on unseen data. In a more general sense, they can hardly achieve optimal compression for individual test images/videos, each of which can in fact be considered a distinct domain. To improve the generalizability, this proposal shall incorporate Noether’s theorem in the form of meta learning to learn an inductive bias that encourages decoded video frames to conserve certain latent consistency in the temporal dimension. We shall also use this learned inductive bias to adapt the encoder and/or the decoder at inference time to suit individual videos. Due to its unsupervised nature, our approach has the striking feature of not having to signal any additional information in the bitstream.
期刊论文(0)
专著(0)
科研奖励(0)
会议论文
Marker-free identification of components for bearing rings
-
批准号:423957182
-
项目类别:Research Grants (Transfer Project)
-
资助金额:$0.0万
-
财政年份:2019
-
负责人:Professor Dr.-Ing. Jörn Ostermann
-
依托单位:
Contour-based Multidirectional Prediction for Intra Coding
-
批准号:397975900
-
项目类别:Research Grants
-
资助金额:$0.0万
-
财政年份:2018
-
负责人:Professor Dr.-Ing. Jörn Ostermann
-
依托单位:
Inductive transfer-learning for the classification of aerial and satellite images using Bayesian methods
-
批准号:246374192
-
项目类别:Research Grants
-
资助金额:$0.0万
-
财政年份:2013
-
负责人:Professor Dr.-Ing. Jörn Ostermann
-
依托单位:
Videorealistische Gesichtsanimation mit natürlichem Gesichtsausdruck für interaktive Dienste
-
批准号:158317137
-
项目类别:Research Grants
-
资助金额:$0.0万
-
财政年份:2009
-
负责人:Professor Dr.-Ing. Jörn Ostermann
-
依托单位:
Planar polymer-optical sensor networks for 2D strain measurement
-
批准号:444745111
-
项目类别:Research Grants
-
资助金额:$0.0万
-
财政年份:--
-
负责人:Professor Dr.-Ing. Jörn Ostermann
-
依托单位:
国内基金
海外基金
登录
查看更多内容
long non-coding RNA(lncRNA)-activatedby TGF-β(lncRNA-ATB)通过成纤维细胞影响糖尿病创面愈合的机制研究
-
批准号:LQ23H150003
-
项目类别:省市级项目
-
资助金额:--
-
批准年份:2023
-
负责人:厉怡
-
依托单位:
Non-coding RNA在RAS抑制剂治疗IgA肾病疗效差异中的作用及机制研究
-
批准号:81770709
-
项目类别:面上项目
-
资助金额:52.0万元
-
批准年份:2017
-
负责人:杨琼琼
-
依托单位:
水稻细菌性褐条病菌致病相关non-coding RNAs的鉴定、功能及调控机制研究
-
批准号:31571971
-
项目类别:面上项目
-
资助金额:60.0万元
-
批准年份:2015
-
负责人:李斌
-
依托单位:
Long non-coding RNA MEG3分子对胶质瘤干细胞调控作用的研究
-
批准号:81402438
-
项目类别:青年科学基金项目
-
资助金额:23.0万元
-
批准年份:2014
-
负责人:胡昳旸
-
依托单位:
调控家蚕发育非编码RNA(non-coding RNA, ncRNA)的功能解析
-
批准号:31172158
-
项目类别:面上项目
-
资助金额:60.0万元
-
批准年份:2011
-
负责人:阚云超
-
依托单位:
保守非基因序列(CNGs),非编码RNA序列(Non-coding RNAs)和内含子(Introns)的信息论研究和功能预测
-
批准号:90403010
-
项目类别:重大研究计划
-
资助金额:25.0万元
-
批准年份:2004
-
负责人:罗辽复
-
依托单位: