This research project falls under the EPSRC Artificial Intelligence Technologies category and specifically focuses on Bayesian Optimization

该研究项目福尔斯EPSRC人工智能技术类别，特别关注贝叶斯优化[1]，人工智能的一个子领域。贝叶斯优化用于优化未知的“黑盒”函数，其中输入和输出之间的关系是未知的。例如，这种“黑盒”函数可以表示电子电路的参数与其速度之间的联系[2]或神经网络架构与其在给定任务上的性能之间的联系[3]。这类问题经常出现在计算机科学和工程中，贝叶斯优化提供了一个合适的框架来解决这些问题。在贝叶斯优化中，第一步是构建输入-输出关系模型，然后用于找到最有希望的输入值。然而，该模型依赖于超参数[4]，这会显著影响其性能。通常，这些超参数是从数据中估计的。然而，在工程问题中，获得新的样本可能是昂贵的。因此，我们掌握的数据通常是有限的，这使得这些估计可能不可靠。到目前为止，还没有一个系统的方法来处理这个问题。认识到文献中存在的差距，我们的项目是由开发一个原则性的方法来选择超参数的目标驱动的“黑盒”函数模型。鉴于有限的数据的约束，我们设想两个潜在的战略来应对这一挑战。这两个研究方向是独立的，我们的项目旨在全面探索这两个方向。一种方法涉及量化与当前超参数估计相关的不确定性。这就需要建立一个理论上合理的方法来估计不确定性，并将这些估计纳入优化过程，使其更加稳健。或者，面对有限的数据，我们可以利用我们以前解决过的类似任务的信息。然而，在此之前，我们需要自动识别哪些过去的任务与我们正在处理的任务具有相似性。大多数现有文献假设超参数是给定的，或者可以很容易地从数据中估计出来。因此，我们的方法的新奇在于解除这一假设，并提出了一个关于如何在更一般的情况下进行贝叶斯优化的基本问题。在整个项目中，我们的目标是开发更强大的算法，能够有效地解决工程问题，并以安全的方式处理超参数中的不确定性。[1]Srinivas，Niranjan等人，“在强盗设置中的高斯过程优化：没有遗憾和实验设计。第27届国际机器学习会议论文集。2010. [2]Grosnit，Antoine，et al.“BOiLS：Bayesian optimization for logic synthesis. 2022年欧洲设计、自动化和测试会议暨展览会（Date）。IEEE，2022年。[3]Nguyen，Vu等人，“顺序和并行神经架构搜索的最佳传输内核。“机器学习国际会议。PMLR，2021年。[4]威廉姆斯，克里斯托弗KI和卡尔爱德华拉斯穆森。机器学习的高斯过程。第二卷。三号。剑桥，MA：MIT出版社，2006年。

This research project falls under the EPSRC Artificial Intelligence Technologies category and specifically focuses on Bayesian Optimization [1], a subfield of AI. Bayesian Optimization is employed for optimizing unknown "black-box" functions, where the relationship between input and output is not known. For instance, such a "black-box" function could represent the connection between the parameters of an electronic circuit and its speed [2] or a neural network architecture and its performance on a given task [3]. These types of problems often arise across computer science and engineering, and Bayesian Optimization provides a suitable framework for addressing them.In Bayesian Optimization, the first step involves constructing a model of the input-output relationship, which is later used to find the most promising input value to try next. However, this model relies on hyperparameters [4], which significantly influence its performance. Typically, these hyperparameters are estimated from data. However, in engineering problems, obtaining new samples can be costly. As such the data at our disposal is usually limited, making these estimates potentially unreliable. Up to this point there has not been a systematic method proposed for dealing with this issue.Recognizing this existing gap in the literature, our project is driven by the goal of developing a principled approach for selecting hyperparameters in "black-box" function models. Given the constraint of limited data, we envision two potential strategies for addressing this challenge. These two research directions are independent, and our project is designed to explore both comprehensively.One approach involves quantifying the uncertainty associated with current hyperparameter estimates. This entails establishing a theoretically sound method for estimating uncertainty and incorporating these estimates into the optimization process to make it more robust. Alternatively, in the face of limited data, we could leverage information from similar tasks that we have previously solved. However, before this could be achieved, we need to automatically identify which past tasks bear similarity to the one we are tackling.A majority of existing literature assumes that hyperparameters are given or can be easily estimated from data. Therefore, the novelty of our methodology lies in lifting this assumption and posing a fundamental question about how to conduct Bayesian Optimization in a more general case. Throughout this project, our goal is to develop more robust algorithms capable of efficiently solving engineering problems and handling the uncertainty in hyperparameters in a safe manner. [1] Srinivas, Niranjan, et al. "Gaussian process optimization in the bandit setting: no regret and experimental design." Proceedings of the 27th International Conference on International Conference on Machine Learning. 2010.[2] Grosnit, Antoine, et al. "BOiLS: Bayesian optimisation for logic synthesis." 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2022.[3] Nguyen, Vu, et al. "Optimal transport kernels for sequential and parallel neural architecture search." International Conference on Machine Learning. PMLR, 2021.[4] Williams, Christopher KI, and Carl Edward Rasmussen. Gaussian processes for machine learning. Vol. 2. No. 3. Cambridge, MA: MIT press, 2006.