Structured Disentangled Representations
Structured Disentangled Representations
复制标题
DOI：
--
复制
发表时间：
2018-04
期刊：
影响因子：
--
通讯作者：
Babak Esmaeili;Hao Wu;Sarthak Jain;Alican Bozkurt;N. Siddharth;Brooks Paige;D. Brooks;Jennifer G. Dy;Jan-Willem van de Meent
中科院分区：
文献类型：
--
作者：
Babak Esmaeili;Hao Wu;Sarthak Jain;Alican Bozkurt;N. Siddharth;Brooks Paige;D. Brooks;Jennifer G. Dy;Jan-Willem van de Meent
深度潜变量模型以无监督的方式学习高维数据的表示。最近的一些努力集中在学习表征上，这些表征通过对标准目标函数进行修改来解开统计独立的变化轴。这些方法通常假设一个简单的对角高斯先验，因此不能可靠地分离离散的变异因素。我们提出了一个两级层次目标来控制变量块之间和块内单个变量的相对统计独立程度。我们将这一目标推导为证据下限的概括，这使我们能够明确地表示数据和表示之间的相互信息之间的权衡，表示和先验之间的KL分歧，以及经验数据分布的支持范围。在各种数据集上的实验表明，我们的目标不仅可以解开离散变量的纠缠，而且这样做还可以改善其他变量的纠缠，更重要的是，甚至可以泛化到看不见的因素组合。
Deep latent-variable models learn representations of high-dimensional data in an unsupervised manner. A number of recent efforts have focused on learning representations that disentangle statistically independent axes of variation by introducing modifications to the standard objective function. These approaches generally assume a simple diagonal Gaussian prior and as a result are not able to reliably disentangle discrete factors of variation. We propose a two-level hierarchical objective to control relative degree of statistical independence between blocks of variables and individual variables within blocks. We derive this objective as a generalization of the evidence lower bound, which allows us to explicitly represent the trade-offs between mutual information between data and representation, KL divergence between representation and prior, and coverage of the support of the empirical data distribution. Experiments on a variety of datasets demonstrate that our objective can not only disentangle discrete variables, but that doing so also improves disentanglement of other variables and, importantly, generalization even to unseen combinations of factors.