Arbitrary Conditional Distributions with Energy
Arbitrary Conditional Distributions with Energy
复制标题
DOI：
--
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
R. Strauss;Junier B. Oliva
中科院分区：
文献类型：
--
作者：
R. Strauss;Junier B. Oliva
协变量的分布建模或密度估计是无监督学习的核心挑战。然而，大多数工作只考虑了联合分布，这在实际情况中的实用性有限。一个更普遍和有用的问题是任意条件密度估计，其目的是在一组协变量上对任何可能的条件分布进行建模，反映基于先验知识的推理的更现实的设置。我们提出了一种新的方法，能量任意条件（ACE），可以同时估计分布$p（\mathbf{x}_u \mid \mathbf{x}_o）$的所有可能的子集的未观察到的功能$\mathbf{x}_u$和观察到的功能$\mathbf{x}_o$。ACE旨在避免不必要的偏差和复杂性-我们用高度表达的能量函数指定密度，并将问题简化为仅学习一维条件（在推理过程中可以从中恢复更复杂的分布）。这导致一种比现有方法更简单且性能更高的方法。我们表明，ACE实现了国家的最先进的任意条件似然估计和数据插补标准基准。
Modeling distributions of covariates, or density estimation, is a core challenge in unsupervised learning. However, the majority of work only considers the joint distribution, which has limited utility in practical situations. A more general and useful problem is arbitrary conditional density estimation, which aims to model any possible conditional distribution over a set of covariates, reflecting the more realistic setting of inference based on prior knowledge. We propose a novel method, Arbitrary Conditioning with Energy (ACE), that can simultaneously estimate the distribution $p(\mathbf{x}_u \mid \mathbf{x}_o)$ for all possible subsets of unobserved features $\mathbf{x}_u$ and observed features $\mathbf{x}_o$. ACE is designed to avoid unnecessary bias and complexity -- we specify densities with a highly expressive energy function and reduce the problem to only learning one-dimensional conditionals (from which more complex distributions can be recovered during inference). This results in an approach that is both simpler and higher-performing than prior methods. We show that ACE achieves state-of-the-art for arbitrary conditional likelihood estimation and data imputation on standard benchmarks.