Predictive Entropy Search for Bayesian Optimization with Unknown Constraints Supplementary Material
Predictive Entropy Search for Bayesian Optimization with Unknown Constraints Supplementary Material
复制标题
具有未知约束的贝叶斯优化的预测熵搜索补充材料
DOI：
--
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Z. Ghahramani
中科院分区：
文献类型：
--
作者：
M. Gelbart;Matthew W. Hoffman;Ryan P. Adams;Z. Ghahramani
PESC使用预期传播（EP）计算对NFCPD（主文本，等式（11））的高斯近似值（Minka，2001）。 EP是一种使用可拖动分布的因素（通常是一个先前因素和多个似然因子）的乘积（例如高斯人）的方法。 EP通过用高斯近似每个因素来生成高斯近似。所有这些高斯人的产品都会产生单个高斯分布，该分布近似于所有确切因素的乘积。这与拉普拉斯的近似相反，该拉普拉斯近似与整个后部拟合了一个高斯分布。可以将EP直观地理解为通过最大程度地减少每个精确因子及其相应的高斯近似之间的kullback-leibler（kl）差异来拟合单个高斯近似值。这对应于精确因素和近似因素之间的第一和第二矩。但是，EP在所有其他近似因素的情况下进行了这一刻，因为我们最终有兴趣在总体后验概率很高的地区具有良好的近似值。具体地，假设我们希望近似分布
PESC computes a Gaussian approximation to the NFCPD (main text, Eq. (11)) using Expectation Propagation (EP) (Minka, 2001). EP is a method for approximating a product of factors (often a single prior factor and multiple likelihood factors) with a tractable distribution, for example a Gaussian. EP generates a Gaussian approximation by approximating each individual factor with a Gaussian. The product all these Gaussians results in a single Gaussian distribution that approximates the product of all the exact factors. This is in contrast to the Laplace approximation which fits a single Gaussian distribution to the whole posterior. EP can be intuitively understood as fitting the individual Gaussian approximations by minimizing the Kullback-Leibler (KL) divergences between each exact factor and its corresponding Gaussian approximation. This would correspond to matching first and second moments between exact and approximate factors. However, EP does this moment matching in the context of all the other approximate factors, since we are ultimately interested in having a good approximation in regions where the overall posterior probability is high. Concretely, assume we wish to approximate the distribution