CAREER: Advancing Efficient Global Optimization of Extremely Expensive Functions under Uncertainty using Structure-Exploiting Bayesian Methods

数学优化是通过在所有可行选项中确定最佳可能值来最大化性能或质量指标的过程。优化问题几乎出现在所有与决策相关的人类活动中，包括工程、经济、可持续性、医疗保健和制造业。每当评估性能和/或可行性测试需要昂贵的模拟或实验，其结果可能被随机错误破坏时，此类优化问题的实例就特别具有挑战性。贝叶斯优化（BO）是一类基于机器学习的优化算法，最近已被证明在该问题类的几个重要应用中实现了最先进的性能，例如深度机器学习，昂贵模拟器的验证以及材料和药物设计。然而，传统的BO方法将模拟性能和可行性的数学函数视为具有未知结构的黑盒子，这从根本上限制了其计算效率。这一观察结果是本研究项目的主要动机，该项目希望通过开发利用贝叶斯框架内已知问题结构的新算法来克服这些效率障碍。这些新功能将应用于目前影响社会的三个未解决的问题：(1)识别生物制造细胞决策过程中的未知机制；(2)发现可持续、经济的新型锂离子电池电极材料；(3)工业供暖、通风和空调（HVAC）系统的实时能源最小化。此外，该项目希望通过开发与决策科学相关的互动讲习班和游戏，将研究和教育活动紧密结合起来，这些活动将向公众开放。通过与当地教育工作者的合作，计划中的外展活动还将为来自代表性不足社区的K-12学生提供学习决策科学的机会。提出的优化方法受到灰盒建模原理的启发，该原理指出，在应用机器学习方法时，应该避免学习已知的内容。研究者推测，通过适当利用在实践中几乎总是可用的先验（或领域）知识，在实践中可以比传统的贝叶斯优化（BO）方法显著减少实验和/或计算工作量。由于先验知识可以有许多不同的形式，因此建议的研究将集中在一些最常见和最重要的例子上。三个具体的研究目标是：(1)在给定噪声和不完整数据集的情况下，使用基于物理和数据驱动的混合模型进行优化；(2)使用约束多保真度模型进行优化，该模型融合了来自不同精度和成本的异构数据源的数据；(3)通过结合非近视和图结构公式，缩放到高维和稀疏数据问题。提出的研究旨在促进统计学，机器学习，优化和过程系统工程的融合。更广泛地说，作为本研究项目的一部分开发的改进方法将允许从业者以更快的速度和准确性解决广泛的灰盒优化问题。计划的推广活动包括通过互动研讨会和游戏教育K-12学生在不确定性下的决策，将新的数据驱动优化材料纳入化学工程课程，以及组织跨学科的专业研讨会，讨论前沿BO技术的潜在意义和影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Mathematical optimization is the process of maximizing a performance or quality indicator by identifying the best possible value among the set of all feasible options. Optimization problems arise in virtually all human endeavors related to decision making including engineering, economics, sustainability, healthcare, and manufacturing. Instances of such optimization problems are particularly challenging to solve whenever evaluating performance and/or testing for feasibility requires an expensive simulation or experiment whose results may be corrupted by random errors. Bayesian optimization (BO) is a class of machine learning-based optimization algorithms that has recently been shown to achieve state-of-the-art performance in several important applications from this problem class such as in deep machine learning, validation of expensive simulators, and material and drug design. However, traditional BO methods treat the mathematical functions that model performance and feasibility as black boxes with unknown structure, which sets a fundamental limit on their computational efficiency. This observation is the key motivation for this research project, which looks to overcome these efficiency barriers via the development of new algorithms that exploit known problem structures within the Bayesian framework. These novel capabilities will be applied to three unsolved problems currently impacting society: (1) identifying unknown mechanisms in cellular decision-making processes for biomanufacturing; (2) discovery of new sustainable and economical lithium-ion battery electrode materials; and (3) real-time energy minimization in industrial heating, ventilation, and air conditioning (HVAC) systems. In addition, the project looks to tightly integrate research and educational activities through the development of interactive workshops and games related to decision science, which will be made accessible to the public. Through collaboration with local educators, planned outreach activities also will provide K-12 students from underrepresented communities with opportunities to learn about decision science.The proposed optimization methodology is inspired by the principle of grey-box modeling, which states that one should avoid learning what is already known when applying machine learning methods. The investigator conjectures a significant reduction in experimental and/or computational effort can be obtained in practice over traditional Bayesian optimization (BO) methods by properly leveraging prior (or domain) knowledge, which is almost always available in practice. Since prior knowledge can come in many diverse forms, the proposed research will focus on some of the most common and important examples. The three specific research aims are: (1) optimizing with hybrid physics-based and data-driven models given noisy and incomplete datasets; (2) optimizing with constrained multi-fidelity models that fuse data from a collection of heterogeneous sources of variable accuracy and cost; and (3) scaling to high-dimensional and sparse data problems through the incorporation of non-myopic and graph-structured formulations. The proposed research aims to promote convergence of statistics, machine learning, optimization, and process systems engineering. More broadly, the improved methods developed as a part of this research project will allow practitioners to solve a wide range of grey-box optimization problems with greater speed and accuracy. Planned outreach activities include educating K-12 students about decision-making under uncertainty via interactive workshops and games, incorporating new data-driven optimization material into the chemical engineering curriculum, and organizing cross-disciplinary professional workshops on the potential significance and impacts of cutting-edge BO technology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.