Construction of Nonparametric Bayesian Models from Parametric Bayes Equations
Construction of Nonparametric Bayesian Models from Parametric Bayes Equations
复制标题
DOI：
--
复制
发表时间：
2009-12
期刊：
影响因子：
--
通讯作者：
Peter Orbanz
中科院分区：
文献类型：
--
作者：
Peter Orbanz
我们考虑在无限维随机对象上构造非参数贝叶斯模型的一般问题，如函数、无限图或无限排列。这个问题在机器学习中引起了很大的兴趣，在机器学习中，它被启发式地对待，但在非参数贝叶斯统计中还没有得到全面的研究，后者倾向于关注概率分布上的模型。我们的方法应用了随机过程理论的一个标准工具，即从随机过程的有限维边缘分布构造随机过程。本文的主要贡献是将经典柯尔莫哥洛夫扩展定理推广到条件概率。这个扩展允许从有限维系统的非参数贝叶斯模型的严格构造，参数贝叶斯方程。使用这种方法，我们展示了(i)如何通过在构造中选择共轭有限维模型来保证非参数模型的共轭后验的存在，（ii）如何显式地确定非参数模型的后验参数的映射，以及（iii）共轭模型的构造本质上要求有限维模型属于指数族。作为构造框架的一个应用，我们推导了一个无限排列的模型，这是最近提出的一个秩数据分析模型的非参数贝叶斯模拟。
We consider the general problem of constructing nonparametric Bayesian models on infinite-dimensional random objects, such as functions, infinite graphs or infinite permutations. The problem has generated much interest in machine learning, where it is treated heuristically, but has not been studied in full generality in non-parametric Bayesian statistics, which tends to focus on models over probability distributions. Our approach applies a standard tool of stochastic process theory, the construction of stochastic processes from their finite-dimensional marginal distributions. The main contribution of the paper is a generalization of the classic Kolmogorov extension theorem to conditional probabilities. This extension allows a rigorous construction of nonparametric Bayesian models from systems of finite-dimensional, parametric Bayes equations. Using this approach, we show (i) how existence of a conjugate posterior for the nonparametric model can be guaranteed by choosing conjugate finite-dimensional models in the construction, (ii) how the mapping to the posterior parameters of the nonparametric model can be explicitly determined, and (iii) that the construction of conjugate models in essence requires the finite-dimensional models to be in the exponential family. As an application of our constructive framework, we derive a model on infinite permutations, the nonparametric Bayesian analogue of a model recently proposed for the analysis of rank data.