An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization
An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization
复制标题
贝叶斯优化自适应核选择的实验研究
DOI：
10.1109/access.2019.2960498
复制
发表时间：
2019
期刊：
影响因子：
3.9
通讯作者：
J. A. Lozano
中科院分区：
文献类型：
--
作者：
Ibai Roman;Roberto Santana;A. Mendiburu;J. A. Lozano
贝叶斯优化与高斯过程一起被广泛用于解决评估代价高昂的黑盒优化问题。总体而言，该方法显示了良好的效果，特别是在机器学习算法的参数调整方面。尽管如此，贝叶斯优化也必须被配置为获得尽可能最佳的性能，这是选择核函数的关键选择。本文研究了在优化过程中自适应改变核函数的方便性，而不是先确定核函数。介绍了六种自适应核选择策略，并在著名的合成和现实世界优化问题中进行了测试。为了对所提出的核选择变量提供更完整的评估，测试了两种主要的核参数设置方法。根据我们的结果，自适应核选择准则除了具有从方程中去掉核选择的优点外，还表现出比固定核方法更好的性能。
Bayesian Optimization has been widely used along with Gaussian Processes for solving expensive-to-evaluate black-box optimization problems. Overall, this approach has shown good results, and particularly for parameter tuning of machine learning algorithms. Nonetheless, Bayesian Optimization has to be also configured to achieve the best possible performance, being the selection of the kernel function a crucial choice. This paper investigates the convenience of adaptively changing the kernel function during the optimization process, instead of fixing it a priori. Six adaptive kernel selection strategies are introduced and tested in well-known synthetic and real-world optimization problems. In order to provide a more complete evaluation of the proposed kernel selection variants, two major kernel parameter setting approaches have been tested. According to our results, apart from having the advantage of removing the selection of the kernel out of the equation, adaptive kernel selection criteria show a better performance than fixed-kernel approaches.