Learning Joint Latent Space EBM Prior Model for Multi-layer Generator
Learning Joint Latent Space EBM Prior Model for Multi-layer Generator
复制标题
DOI：
10.1109/cvpr52729.2023.00351
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Jiali Cui;Y. Wu;Tian Han
中科院分区：
文献类型：
--
作者：
Jiali Cui;Y. Wu;Tian Han
本文研究了多层发电机模型学习的基本问题。多层生成器模型在生成器之上构建多层潜在变量作为先验模型，这有利于学习复杂的数据分布和分层表示。然而，这样的先验模型通常通过假设非信息（条件）高斯分布来对潜在变量之间的层间关系进行建模，这可能在模型表达能力方面受到限制。为了解决这个问题，并学习更有表现力的先验模型，我们提出了一个基于能量的模型（EBM）的联合潜在空间的所有层的潜变量与多层生成器作为其骨干。这种联合潜空间EBM先验模型通过逐层能量项捕获每层的层内上下文关系，并且跨不同层的潜变量被联合校正。我们通过最大似然估计（MLE）开发了一个联合训练方案，该方案涉及马尔可夫链蒙特卡罗（MCMC）对来自不同层的潜变量的先验和后验分布进行采样。为了确保有效的推理和学习，我们进一步提出了一个变分训练方案，其中推理模型用于摊销昂贵的后验MCMC采样。我们的实验表明，学习的模型可以在生成高质量的图像和捕获层次特征，以更好地离群点检测表达。
This paper studies the fundamental problem of learning multi-layer generator models. The multi-layer generator model builds multiple layers of latent variables as a prior model on top of the generator, which benefits learning complex data distribution and hierarchical representations. However, such a prior model usually focuses on modeling inter-layer relations between latent variables by assuming non-informative (conditional) Gaussian distributions, which can be limited in model expressivity. To tackle this issue and learn more expressive prior models, we propose an energy-based model (EBM) on the joint latent space over all layers of latent variables with the multi-layer generator as its backbone. Such joint latent space EBM prior model captures the intra-layer contextual relations at each layer through layer-wise energy terms, and latent variables across different layers are jointly corrected. We develop a joint training scheme via maximum likelihood estimation (MLE), which involves Markov Chain Monte Carlo (MCMC) sampling for both prior and posterior distributions of the latent variables from different layers. To ensure efficient inference and learning, we further propose a variational training scheme where an inference model is used to amortize the costly posterior MCMC sampling. Our experiments demonstrate that the learned model can be expressive in generating high-quality images and capturing hierarchical features for better outlier detection.