A Generalized Framework of Multi-fidelity Max-value Entropy Search through Joint Entropy
A Generalized Framework of Multi-fidelity Max-value Entropy Search through Joint Entropy
复制标题
通过联合熵进行多保真极大值熵搜索的通用框架
DOI：
10.1162/neco_a_01530
复制
发表时间：
2022
影响因子：
2.9
通讯作者：
I. Takeuchi and M. Karasuyama
中科院分区：
文献类型：
--
作者：
S. Takeno;H. Fukuoka;Y. Tsukada;T. Koyama;M Shiga;I. Takeuchi and M. Karasuyama
贝叶斯优化（BO）是一种用于昂贵的黑盒优化问题的流行方法;然而，在每次迭代时查询目标函数可能是阻碍有效搜索能力的瓶颈。在这方面，多保真度贝叶斯优化（MFBO）的目的是加速BO结合较低的保真度观察可用较低的采样成本。在我们以前的工作中，我们提出了一个信息理论的方法MFBO，称为多保真度最大值熵搜索（MF-MES），它继承了著名的单保真度BO的最大值熵搜索（MES）的实际有效性和计算简单性。然而，MF-MES的适用性仍然限于顺序获得单个观测的情况。在这封信中，我们推广MF-MES，使信息增益可以评估，即使当多个观测同时获得。这种泛化使MF-MES能够解决两个实际问题设置：同步并行化和跟踪感知查询。我们发现，这些扩展的采集功能继承了MF-MES的简单性，而无需引入额外的假设。我们还提供了计算技术的熵评估和后验采样的采集功能，这可以通常用于所有的MF-MES的变种。MF-MES的有效性使用基准函数和真实世界的应用程序，如材料科学数据和机器学习算法的超参数调整。
Bayesian optimization (BO) is a popular method for expensive black-box optimization problems; however, querying the objective function at every iteration can be a bottleneck that hinders efficient search capabilities. In this regard, multifidelity Bayesian optimization (MFBO) aims to accelerate BO by incorporating lower-fidelity observations available with a lower sampling cost. In our previous work, we proposed an information-theoretic approach to MFBO, referred to as multifidelity max-value entropy search (MF-MES), which inherits practical effectiveness and computational simplicity of the well-known max-value entropy search (MES) for the single-fidelity BO. However, the applicability of MF-MES is still limited to the case that a single observation is sequentially obtained. In this letter, we generalize MF-MES so that information gain can be evaluated even when multiple observations are simultaneously obtained. This generalization enables MF-MES to address two practical problem settings: synchronous parallelization and trace-aware querying. We show that the acquisition functions for these extensions inherit the simplicity of MF-MES without introducing additional assumptions. We also provide computational techniques for entropy evaluation and posterior sampling in the acquisition functions, which can be commonly used for all variants of MF-MES. The effectiveness of MF-MES is demonstrated using benchmark functions and real-world applications such as materials science data and hyperparameter tuning of machine-learning algorithms.