Surrogate Modeling for Bayesian Optimization Beyond a Single Gaussian Process
Surrogate Modeling for Bayesian Optimization Beyond a Single Gaussian Process
复制标题
DOI：
10.1109/tpami.2023.3264741
复制
发表时间：
2022-05
影响因子：
23.6
通讯作者：
Qin Lu;Konstantinos D. Polyzos;Bingcong Li;G. Giannakis
中科院分区：
文献类型：
--
作者：
Qin Lu;Konstantinos D. Polyzos;Bingcong Li;G. Giannakis
贝叶斯优化（BO）具有有据可查的优点，可通过昂贵的评估成本优化黑盒功能。这些功能在多种多样的应用程序调整，药物发现和机器人技术等应用中出现。 BO HUTES在贝叶斯替代模型上，以顺序选择查询点，以平衡探索与搜索空间的开发。大多数现有作品都依赖于基于单个高斯过程（GP）的替代模型，其中通常使用域知识预选内核函数形式。为了绕过这样的设计过程，本文利用GPS的集合（E）自适应地选择了替代模型，从而在fly上适合拟合，从而产生了GP的后验，并具有增强的表达能力。然后，通过不需要其他设计参数的汤普森采样（TS）来启用使用此基于EGP的功能后验的下一个评估输入。为了赋予具有可扩展性的功能采样，根据GP模型，基于特征的内核近似是利用的。新颖的EGP-TS很容易适应并行操作。为了进一步建立所提出的EGP-TS与全局最佳量的收敛性，分析是基于贝叶斯对顺序和平行设置的贝叶斯后悔的概念进行的。关于合成功能和现实世界应用的测试展示了所提出方法的优点。
Bayesian optimization (BO) has well-documented merits for optimizing black-box functions with an expensive evaluation cost. Such functions emerge in applications as diverse as hyperparameter tuning, drug discovery, and robotics. BO hinges on a Bayesian surrogate model to sequentially select query points so as to balance exploration with exploitation of the search space. Most existing works rely on a single Gaussian process (GP) based surrogate model, where the kernel function form is typically preselected using domain knowledge. To bypass such a design process, this paper leverages an ensemble (E) of GPs to adaptively select the surrogate model fit on-the-fly, yielding a GP mixture posterior with enhanced expressiveness for the sought function. Acquisition of the next evaluation input using this EGP-based function posterior is then enabled by Thompson sampling (TS) that requires no additional design parameters. To endow function sampling with scalability, random feature-based kernel approximation is leveraged per GP model. The novel EGP-TS readily accommodates parallel operation. To further establish convergence of the proposed EGP-TS to the global optimum, analysis is conducted based on the notion of Bayesian regret for both sequential and parallel settings. Tests on synthetic functions and real-world applications showcase the merits of the proposed method.