Quantile Stein Variational Gradient Descent for Batch Bayesian Optimization
Quantile Stein Variational Gradient Descent for Batch Bayesian Optimization
复制标题
DOI：
--
复制
发表时间：
2019-05
期刊：
影响因子：
2.4
通讯作者：
Chengyue Gong;Jian Peng;Qiang Liu
中科院分区：
文献类型：
--
作者：
Chengyue Gong;Jian Peng;Qiang Liu
批处理贝叶斯优化已被证明是一种有效和成功的黑盒函数优化方法，特别是当代价函数的计算代价很高但可以高效并行化的时候。本文提出了一种新的批查询优化变分框架，该框架认为查询批应该同时具有高的多样性和良好的最坏情况性能。这促使我们引入一个变分目标，该目标结合了基于分位数的风险度量(对于最差的情况表现)和熵正则化(用于加强多样性)。我们推导了一种基于梯度的粒子优化算法来求解我们的基于分位数的变分目标，推广了Liu&Wang(2016)的Stein变分梯度下降(SVGD)算法。我们在实际应用中对我们的方法进行了评估，结果表明，它的性能始终优于其他最新的批处理贝叶斯优化方法。
Batch Bayesian optimization has been shown to be an efficient and successful approach for blackbox function optimization, especially when the evaluation of cost function is highly expensive but can be efficiently parallelized. In this paper, we introduce a novel variational framework for batch query optimization, based on the argument that the query batch should be selected to have both high diversity and good worst case performance. This motivates us to introduce a variational objective that combines a quantile-based risk measure (for worst case performance) and entropy regularization (for enforcing diversity). We derive a gradient-based particle optimization algorithm for solving our quantile-based variational objective, which generalizes Stein variational gradient descent (SVGD) by Liu & Wang (2016). We evaluate our method on a number of real-world applications, and show that it consistently outperforms other recent state-of-the-art batch Bayesian optimization methods.