Expected utility estimation via cross-validation
Expected utility estimation via cross-validation
复制标题
DOI：
--
复制
发表时间：
2003
期刊：
影响因子：
--
通讯作者：
Aki Vehtari;J. Lampinen
中科院分区：
文献类型：
--
作者：
Aki Vehtari;J. Lampinen
总结 我们讨论了评估、比较和选择复杂分层贝叶斯模型的实用方法。评估模型优劣的一种自然方法是通过估计预期效用来估计其未来的预测能力。重要的是获得预期效用估计的分布以描述相关的不确定性，而不仅仅是进行点估计。我们通过多种方式综合和扩展了之前的工作。我们从贝叶斯观点进行统一的介绍，强调所做的假设，并提出获得预期效用估计分布的实用方法。我们讨论了两种实用方法的特性，即重要性抽样留一法和 k 折交叉验证。我们提出了一种基于贝叶斯引导的快速通用方法，用于从预期效用估计的分布中获取样本。这些分布还可用于模型比较，例如，通过计算一个模型比其他模型具有更好预期效用的概率。我们讨论交叉验证方法与其他预测密度方法的不同之处，以及交叉验证与信息标准方法的关系，这也可用于估计预期效用。我们用一个玩具和两个现实世界的例子来说明讨论。
SUMMARY We discuss practical methods for the assessment, comparison and selection of complex hierarchical Bayesian models. A natural way to assess the goodness of the model is to estimate its future predictive capability by estimating expected utilities. Instead of just making a point estimate, it is important to obtain the distribution of the expected utility estimate in order to describe the associated uncertainty. We synthesize and extend the previous work in several ways. We give a unified presentation from the Bayesian viewpoint emphasizing the assumptions made and propose practical methods to obtain the distributions of the expected utility estimates. We discuss the properties of two practical methods, the importance sampling leave-one-out and the k-fold cross-validation. We propose a quick and generic approach based on the Bayesian bootstrap for obtaining samples from the distributions of the expected utility estimates. These distributions can also be used for model comparison, for example, by computing the probability of one model having a better expected utility than some other model. We discuss how the crossvalidation approach differs from other predictive density approaches, and the relationship of cross-validation to information criteria approaches, which can also be used to estimate the expected utilities. We illustrate the discussion with one toy and two real world examples.