Local Latent Space Bayesian Optimization over Structured Inputs
Local Latent Space Bayesian Optimization over Structured Inputs
复制标题
DOI：
--
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
N. Maus;Haydn Jones;Juston Moore;Matt J. Kusner;John Bradshaw;J. Gardner
中科院分区：
文献类型：
--
作者：
N. Maus;Haydn Jones;Juston Moore;Matt J. Kusner;John Bradshaw;J. Gardner
深度自动编码器模型（DAE）的潜在空间上的贝叶斯优化最近已经成为一种有前途的新方法，用于在结构化、离散、难以枚举的搜索空间（例如，分子）。这里，DAE通过将输入映射到连续的潜在空间中，可以更容易地应用熟悉的贝叶斯优化工具，从而大大简化了搜索空间。尽管这种简化，潜在空间通常仍然是高维的。因此，即使有一个非常合适的潜在空间，这些方法并不一定提供一个完整的解决方案，但可能会转移到一个高维的结构化优化问题。在本文中，我们提出了LOL-BO，它适应了最近的高维贝叶斯优化工作中探索的信任区域的概念结构化设置。通过重新定义编码器，使其既作为DAE的全局编码器，又作为信任区域内代理模型的深层内核，我们可以更好地将潜在空间中的局部优化概念与输入空间中的局部优化相结合。LOL-BO在六个真实世界的基准测试中实现了最先进的潜在空间贝叶斯优化方法的20倍改进，表明优化策略的改进与开发更好的DAE模型一样重要。
Bayesian optimization over the latent spaces of deep autoencoder models (DAEs) has recently emerged as a promising new approach for optimizing challenging black-box functions over structured, discrete, hard-to-enumerate search spaces (e.g., molecules). Here the DAE dramatically simplifies the search space by mapping inputs into a continuous latent space where familiar Bayesian optimization tools can be more readily applied. Despite this simplification, the latent space typically remains high-dimensional. Thus, even with a well-suited latent space, these approaches do not necessarily provide a complete solution, but may rather shift the structured optimization problem to a high-dimensional one. In this paper, we propose LOL-BO, which adapts the notion of trust regions explored in recent work on high-dimensional Bayesian optimization to the structured setting. By reformulating the encoder to function as both an encoder for the DAE globally and as a deep kernel for the surrogate model within a trust region, we better align the notion of local optimization in the latent space with local optimization in the input space. LOL-BO achieves as much as 20 times improvement over state-of-the-art latent space Bayesian optimization methods across six real-world benchmarks, demonstrating that improvement in optimization strategies is as important as developing better DAE models.