A hierarchical expected improvement method for Bayesian optimization
A hierarchical expected improvement method for Bayesian optimization
复制标题
DOI：
10.1080/01621459.2023.2210803
复制
发表时间：
2019-11
影响因子：
3.7
通讯作者：
Zhehui Chen;Simon Mak;C. F. J. Wu
中科院分区：
文献类型：
--
作者：
Zhehui Chen;Simon Mak;C. F. J. Wu
摘要由Jones、Schonlau和Welch提出的期望改进(EI)方法是一种广泛使用的贝叶斯优化方法，它利用一个拟合的高斯过程模型进行有效的黑盒优化。然而，EI的一个关键缺陷是它在利用拟合的高斯过程模型进行优化时过于贪婪，这导致即使在大样本情况下也会产生次优解。为了解决这一问题，我们提出了一种新的分层EI(HEI)框架，该框架利用了分层高斯过程模型。HEI保留了封闭形式的获取函数，并通过鼓励对优化空间的探索来纠正EI的过度贪婪。然后，我们介绍了超参数估计方法，这些方法允许HEI模拟完全贝叶斯优化过程，同时避免昂贵的马尔可夫链蒙特卡罗抽样步骤。我们证明了HEI在广义函数空间上的全局收敛，并在一定的先验条件下建立了近极小极大收敛速度。数值实验表明，对于合成函数和一个半导体制造优化问题，HEI比现有的贝叶斯优化方法有更好的性能。这篇文章的补充材料可以在网上找到。
Abstract The Expected Improvement (EI) method, proposed by Jones, Schonlau, andWelch, is a widely used Bayesian optimization method, which makes use of a fitted Gaussian process model for efficient black-box optimization. However, one key drawback of EI is that it is overly greedy in exploiting the fitted Gaussian process model for optimization, which results in suboptimal solutions even with large sample sizes. To address this, we propose a new hierarchical EI (HEI) framework, which makes use of a hierarchical Gaussian process model. HEI preserves a closed-form acquisition function, and corrects the over-greediness of EI by encouraging exploration of the optimization space. We then introduce hyperparameter estimation methods which allow HEI to mimic a fully Bayesian optimization procedure, while avoiding expensive Markov-chain Monte Carlo sampling steps. We prove the global convergence of HEI over a broad function space, and establish near-minimax convergence rates under certain prior specifications. Numerical experiments show the improvement of HEI over existing Bayesian optimization methods, for synthetic functions and a semiconductor manufacturing optimization problem. Supplementary materials for this article are available online.