Cost-aware Bayesian Optimization
Cost-aware Bayesian Optimization
复制标题
成本感知贝叶斯优化
DOI：
10.1145/3299904.3340307
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
M. Seeger
中科院分区：
文献类型：
--
作者：
E. Lee;Valerio Perrone;C. Archambeau;M. Seeger
贝叶斯优化（BO）是一类全局优化算法，适用于在尽可能少的函数评估中最小化昂贵的目标函数。虽然BO预算通常在迭代中给出，但这隐含地根据迭代计数来度量收敛，并假设每个评估具有相同的成本。在实践中，评估成本在搜索空间的不同区域可能会有所不同。例如，神经网络训练的成本与层大小成二次方增加，这是一个典型的超参数。成本感知的BO测量与时间、精力或金钱等替代成本指标的融合，普通BO方法不适合这些成本指标。我们引入成本分摊BO（CArBO），它试图以尽可能少的成本最小化目标函数。CArBO结合了成本效益的初始设计与成本冷却优化阶段，随着迭代的进行，该阶段会使学习到的成本模型贬值。在一组20黑盒函数优化问题，我们表明，在相同的成本预算，CArBO发现显着更好的超参数配置比竞争的方法。
Bayesian optimization (BO) is a class of global optimization algorithms, suitable for minimizing an expensive objective function in as few function evaluations as possible. While BO budgets are typically given in iterations, this implicitly measures convergence in terms of iteration count and assumes each evaluation has identical cost. In practice, evaluation costs may vary in different regions of the search space. For example, the cost of neural network training increases quadratically with layer size, which is a typical hyperparameter. Cost-aware BO measures convergence with alternative cost metrics such as time, energy, or money, for which vanilla BO methods are unsuited. We introduce Cost Apportioned BO (CArBO), which attempts to minimize an objective function in as little cost as possible. CArBO combines a cost-effective initial design with a cost-cooled optimization phase which depreciates a learned cost model as iterations proceed. On a set of 20 black-box function optimization problems we show that, given the same cost budget, CArBO finds significantly better hyperparameter configurations than competing methods.
影响因子：
14.4
作者：
Hutter, Frank;Xu, Lin;Leyton-Brown, Kevin
通讯作者：
Leyton-Brown, Kevin