Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound
Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound
复制标题
DOI：
--
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Shion Takeno;T. Tamura;Kazuki Shitara;Masayuki Karasuyama
中科院分区：
文献类型：
--
作者：
Shion Takeno;T. Tamura;Kazuki Shitara;Masayuki Karasuyama
最大值熵搜索(MES)是贝叶斯优化(BO)中最先进的方法之一。本文提出了一种基于互信息下界蒙特卡罗(MC)估计的约束MES的新变种，称为基于信息下界的约束MES(CMES-IBO)。与现有研究不同的是，我们的MI的定义是这样的，即可以纳入关于可行性的不确定性。我们得到了保证非负性的MI的下界，而传统MES的约束对应项可以是负的。我们进一步提供了理论分析，以确保我们的估计量的低变异性，这是任何现有的信息论BO从未研究过的。此外，使用条件MI，我们将CMES-IBO扩展到并行环境中，同时保持了期望的性质。我们通过几个基准函数和实际问题验证了CMES-IBO的有效性。
Max-value entropy search (MES) is one of the state-of-the-art approaches in Bayesian optimization (BO). In this paper, we propose a novel variant of MES for constrained problems, called Constrained MES via Information lower BOund (CMES-IBO), that is based on a Monte Carlo (MC) estimator of a lower bound of a mutual information (MI). Unlike existing studies, our MI is defined so that uncertainty with respect to feasibility can be incorporated. We derive a lower bound of the MI that guarantees non-negativity, while a constrained counterpart of conventional MES can be negative. We further provide theoretical analysis that assures the low-variability of our estimator which has never been investigated for any existing information-theoretic BO. Moreover, using the conditional MI, we extend CMES-IBO to the parallel setting while maintaining the desirable properties. We demonstrate the effectiveness of CMES-IBO by several benchmark functions and real-world problems.