Bayesian Optimization in a Billion Dimensions via Random Embeddings
Bayesian Optimization in a Billion Dimensions via Random Embeddings
复制标题
DOI：
10.1613/jair.4806
复制
发表时间：
2013-01
期刊：
影响因子：
--
通讯作者：
Ziyun Wang;M. Zoghi;F. Hutter;David Matheson;Nando de Freitas
中科院分区：
文献类型：
--
作者：
Ziyun Wang;M. Zoghi;F. Hutter;David Matheson;Nando de Freitas
贝叶斯优化技术已成功应用于机器人、规划、传感器放置、推荐、广告、智能用户界面和自动算法配置。尽管取得了这些成功，但这种方法仅限于中等维度的问题，贝叶斯优化的几个研讨会已经将其扩展到高维作为该领域的圣杯之一。在本文中，我们引入了一种新的随机嵌入的想法来解决这个问题。由此产生的随机嵌入贝叶斯优化（REMBO）算法是非常简单的，具有重要的不变性，并适用于域的分类和连续变量。本文对REMBO进行了深入的理论分析。实证结果表明，REMBO可以有效地解决数十亿维的问题，只要内在维数低。他们还表明，REMBO在优化流行的混合整数线性规划求解器的47个离散参数方面达到了最先进的性能。
Bayesian optimization techniques have been successfully applied to robotics, planning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach is restricted to problems of moderate dimension, and several workshops on Bayesian optimization have identified its scaling to high-dimensions as one of the holy grails of the field. In this paper, we introduce a novel random embedding idea to attack this problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple, has important invariance properties, and applies to domains with both categorical and continuous variables. We present a thorough theoretical analysis of REMBO. Empirical results confirm that REMBO can effectively solve problems with billions of dimensions, provided the intrinsic dimensionality is low. They also show that REMBO achieves state-of-the-art performance in optimizing the 47 discrete parameters of a popular mixed integer linear programming solver.