CAREER: Ensemble Kalman Methods and Bayesian Optimization in Inverse Problems and Data Assimilation

将复杂的预测模型与数据混合在许多应用中是必不可少的，包括数值天气预报、气候科学、石油工程、信号处理和医学成像。正演模型和动力系统在反问题和数据同化中日益复杂所带来的挑战可以通过开发无导数且需要很少模型评估的计算方法来缓解。本项目涉及两个重要的成本效益，无导数的算法家族：集成卡尔曼方法和贝叶斯优化。将建立严格的数学分析，这将有助于理解这些算法，确定它们在高维反问题和数据同化中的潜力和局限性。方法学方面的贡献将集中在新算法和计算框架的设计上，以合并无导数优化与机器学习。这些新算法的数值实现将会公开。除了逆问题和数据同化之外，首席研究员还将研究集成卡尔曼方法和贝叶斯优化在大规模科学计算问题中的潜力，这些问题的梯度不可用或计算成本昂贵，以及隐私问题的数据科学应用。该项目的一个核心组成部分是教育和研究的整合。首席研究员将参与新的导师项目，这是一个在芝加哥少数族裔社区大学建立数据科学课程的合作倡议。该项目将为来自弱势群体的社区大学学生提供转学到芝加哥大学的途径。此外，研究者将完成两本针对研究生和高年级本科生的书，这些书将包含从本项目中提取的主题。研究生的指导和研究生水平课程的开发是该项目的核心部分。该项目将包括两个相互关联的研究重点：集成卡尔曼方法和贝叶斯优化。集合卡尔曼方法是地球物理科学中常用的算法，通常用于较小的集合规模，以保持较低的模型评估数量。该项目的第一个研究重点将开发一种新的综合非渐近分析系综卡尔曼方法，严格解释何时以及为什么小系综规模可能足够。以前的分析集中在大集合渐近性上，不能解释这些算法在小集合规模下的实际成功。本研究的方法学贡献将集中在推导混合集成卡尔曼方法和机器学习的原则框架，以及基于逆问题和数据同化的分层公式的新正则化技术。提出的非渐近理论将为这些新方法建立集合大小要求。该项目的第二个研究重点将通过开发新的几何感知核、获取函数和收敛保证来推进图形和流形设置中的贝叶斯优化。PI将使用计算调和分析的工具来获得流形上随机过程的近似保证，并使用信息论的工具来获得错误指定模型下的遗憾界。最后，研究者将探索将集成卡尔曼方法和贝叶斯优化相结合的协同方法，利用这两个算法家族的优势来减轻它们的弱点。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Blending complex predictive models with data is essential in many applications, including numerical weather forecasting, climate science, petroleum engineering, signal processing, and medical imaging. The challenges posed by the increasing complexity of forward models and dynamical systems in inverse problems and data assimilation can be mitigated by the development of computational methods that are derivative-free and require few model evaluations. This project is concerned with two important families of cost-efficient, derivative-free algorithms: ensemble Kalman methods and Bayesian optimization. Rigorous mathematical analyses will established which will contribute to the understanding of these algorithms, determining their potential and limitations in high-dimensional inverse problems and data assimilation. Methodological contributions will focus on the design of novel algorithms and computational frameworks to merge derivative-free optimization with machine learning. Numerical implementations of these new algorithms will be made publicly available. Beyond inverse problems and data assimilation, the principal investigator will also investigate the potential of ensemble Kalman methods and Bayesian optimization in large-scale scientific computing problems where gradients are unavailable or expensive to compute, and in data science applications where privacy is a concern. A central component of the project is the integration of education and research. The principal investigator will engage in the new Preceptor Program, a collaborative initiative to build data science curricula at minority-serving community colleges in Chicago. This program will create pathways for community college students from underrepresented groups to transfer to the University of Chicago. In addition, the investigator will complete two books aimed at graduate and upper-level undergraduate students that will incorporate topics drawn from this project. Mentorship of graduate students and development of graduate-level courses is a core part of the project.The project will consist of two interrelated research thrusts on ensemble Kalman methods and Bayesian optimization. Ensemble Kalman methods are popular algorithms in the geophysical sciences, where they are often used with a small ensemble size to keep the number of model evaluations low. The first research thrust of the project will develop a new comprehensive non-asymptotic analysis of ensemble Kalman methods that rigorously explains when and why a small ensemble size may suffice. Previous analyses have focused instead on large ensemble asymptotics that cannot explain the practical success of these algorithms with a small ensemble size. Methodological contributions of this research thrust will be focused on deriving principled frameworks to blend ensemble Kalman methods and machine learning, as well as novel regularization techniques based on hierarchical formulations of inverse problems and data assimilation. The proposed non-asymptotic theory will establish ensemble size requirements for these new methods. The second research thrust of the project will advance Bayesian optimization in graphical and manifold settings by developing new geometry-aware kernels, acquisition functions, and convergence guarantees. The PI will use tools from computational harmonic analysis to obtain approximation guarantees for stochastic processes on manifolds and tools from information theory to obtain regret bounds under mis-specified models. Finally, the investigator will explore synergistic ways to combine ensemble Kalman methods and Bayesian optimization, leveraging the strengths of both families of algorithms to mitigate their weaknesses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.