On Tractable Computation of Expected Predictions
On Tractable Computation of Expected Predictions
复制标题
DOI：
--
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Pasha Khosravi;YooJung Choi;Yitao Liang;Antonio Vergari;Guy Van den Broeck
中科院分区：
文献类型：
--
作者：
Pasha Khosravi;YooJung Choi;Yitao Liang;Antonio Vergari;Guy Van den Broeck
计算判别模型的预期预测是机器学习中的一项基本任务，它出现在许多有趣的应用中，例如公平，处理丢失值和数据分析。不幸的是，证明对由任意生成模型定义的概率分布的计算判别模型的计算期望已被证明很难。实际上，即使对于简单的模型，例如逻辑回归和天真的贝叶斯分布，任务也很棘手。在本文中，我们确定了一对生成和歧视性模型，这些模型可以实现后者相对于前者进行回归的后者的期望以及任何顺序的矩。具体而言，我们考虑具有某些结构性约束的表达性概率回路，这些结构性约束支持可进行的概率推断。此外，我们利用高阶矩的可拖动计算来得出算法，以近似对精确计算非常棘手的分类场景的期望。我们计算预期预测的框架可以以原则上准确的方式在预测时间内处理丢失的数据，并可以推理有关判别模型的行为。我们从经验上表明，我们的算法在各种数据集上始终超过标准插补技术。最后，我们说明了如何将我们的框架用于探索性数据分析。
Computing expected predictions of discriminative models is a fundamental task in machine learning that appears in many interesting applications such as fairness, handling missing values, and data analysis. Unfortunately, computing expectations of a discriminative model with respect to a probability distribution defined by an arbitrary generative model has been proven to be hard in general. In fact, the task is intractable even for simple models such as logistic regression and a naive Bayes distribution. In this paper, we identify a pair of generative and discriminative models that enables tractable computation of expectations, as well as moments of any order, of the latter with respect to the former in case of regression. Specifically, we consider expressive probabilistic circuits with certain structural constraints that support tractable probabilistic inference. Moreover, we exploit the tractable computation of high-order moments to derive an algorithm to approximate the expectations for classification scenarios in which exact computations are intractable. Our framework to compute expected predictions allows for handling of missing data during prediction time in a principled and accurate way and enables reasoning about the behavior of discriminative models. We empirically show our algorithm to consistently outperform standard imputation techniques on a variety of datasets. Finally, we illustrate how our framework can be used for exploratory data analysis.