Ladder Variational Autoencoders
Ladder Variational Autoencoders
复制标题
DOI：
--
复制
发表时间：
2016-02
期刊：
影响因子：
--
通讯作者：
C. Sønderby;T. Raiko;Lars Maaløe;Søren Kaae Sønderby;O. Winther
中科院分区：
文献类型：
--
作者：
C. Sønderby;T. Raiko;Lars Maaløe;Søren Kaae Sønderby;O. Winther
各种自动编码器是无监督学习的强大模型。但是，具有几层依赖性随机变量的深层模型很难训练，这限制了使用这些高度表达模型获得的改进。我们提出了一种新的推理模型，即梯子变分自动编码器，该模型在类似于最近提出的梯子网络的过程中，通过数据相关的近似可能性递归纠正生成分布。我们表明，与层次的变分自动编码器和其他生成模型的纯粹自下而上的推断相比，该模型提供了最先进的对数可能性和更紧密的对数可能性下限的状态。我们提供了对学到的层次潜在表示的详细分析，并表明我们的新推论模型在质量上有所不同，并利用了潜在变量的更深层次的分布层次结构。最后，我们观察到批归一化和确定性热身（逐渐打开KL期）对于具有许多随机层的训练变异模型至关重要。
Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that recursively corrects the generative distribution by a data dependent approximate likelihood in a process resembling the recently proposed Ladder Network. We show that this model provides state of the art predictive log-likelihood and tighter log-likelihood lower bound compared to the purely bottom-up inference in layered Variational Autoencoders and other generative models. We provide a detailed analysis of the learned hierarchical latent representation and show that our new inference model is qualitatively different and utilizes a deeper more distributed hierarchy of latent variables. Finally, we observe that batch normalization and deterministic warm-up (gradually turning on the KL-term) are crucial for training variational models with many stochastic layers.