Global Optimization with Parametric Function Approximation-猫眼课题宝

DOI：

10.48550/arxiv.2211.09100

发表时间：

2022-11

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Chong Liu;Yu-Xiang Wang

我们考虑使用嘈杂的零阶预言机进行全局优化的问题——这是一个动机良好的问题，适用于从深度学习的超参数调整到新材料设计的各种应用。现有的工作依赖于高斯过程或其他非参数族，它们受到维数灾难的影响。在本文中，我们提出了一种新算法 GO-UCB，它利用参数函数族（例如神经网络）。在可实现的假设和其他一些温和的几何条件下，我们表明 GO-UCB 实现了 \~O$(\sqrt{T})$ 的累积遗憾，其中 $T$ 是时间范围。 GO-UCB 的核心是基于梯度的参数的精心设计的不确定性集，允许乐观探索。综合和现实世界的实验表明，即使模型指定错误，GO-UCB 也比流行的贝叶斯优化方法效果更好。

We consider the problem of global optimization with noisy zeroth order oracles - a well-motivated problem useful for various applications ranging from hyper-parameter tuning for deep learning to new material design. Existing work relies on Gaussian processes or other non-parametric family, which suffers from the curse of dimensionality. In this paper, we propose a new algorithm GO-UCB that leverages a parametric family of functions (e.g., neural networks) instead. Under a realizable assumption and a few other mild geometric conditions, we show that GO-UCB achieves a cumulative regret of \~O$(\sqrt{T})$ where $T$ is the time horizon. At the core of GO-UCB is a carefully designed uncertainty set over parameters based on gradients that allows optimistic exploration. Synthetic and real-world experiments illustrate GO-UCB works better than popular Bayesian optimization approaches, even if the model is misspecified.

Global Optimization with Parametric Function Approximation

文献摘要

相似文献