Collaborative Research: Frameworks: Convergence of Bayesian inverse methods and scientific machine learning in Earth system models through universal differentiable programming

理解和量化模拟系统的参数敏感性，如物理系统的数值模型和神经网络的数学渲染，在基于模拟的科学（SBS）和科学机器学习（scil）中是必不可少的。它们是贝叶斯推理和神经网络训练的关键成分。利用Julia高级编程语言开发新兴的开源地球系统模型的机会，该项目赋予这些开源模型自动区分（AD）衍生信息，使这些融合数据科学和基于仿真的科学工具可用于更广泛的研究和数据科学社区。启用通用AD框架，既可以处理大规模地球系统模型，也可以处理SciML算法，如物理信息神经网络或神经微分方程，将这些方法无缝集成到混合贝叶斯反演和贝叶斯机器学习中。它融合了大数据科学，其中可用数据使模型发现与稀疏数据科学相结合，并且模型结构被用于选择代表数据知情子空间并满足守恒定律的代理模型。新兴的Julia语言吸引了新一代的研究人员和软件工程师，为气候建模的计算科学方法引入了急需的人才。通过专门的社区外展计划（例如，黑客马拉松，迷你研讨会，教程），项目团队将致力于增加参与学科的公平性，多样性和包容性。该项目正在开发一个通用可微分编程和开源通用AD的框架，该框架将这些算法框架统一在Julia编程语言中。Julia中的通用AD框架利用了Julia软件包的可组合性和许多SciML和高性能科学计算包的可微分编程方法。与目前大多数针对HPC的建模系统相比，Julia非常适合异构并行计算硬件（例如，CUDA, ROCm, oneAPI， ARM, PowerPC, x86 64, tpu）。该项目汇集了针对高性能计算环境中地球系统数据同化的AD专业知识与scil专业知识。项目团队正在与Julia Computing组织和包开发人员合作，以确保开发框架的可持续性。该项目的地球系统旗舰应用程序包括：(1)作为气候建模联盟（CliMA）的一部分，正在单独开发的开源、支持人工智能的海洋环流模型；(2)开源、支持人工智能的冰流模型。这些应用程序框架中的每一个都可供科学应用社区使用，其中衍生（梯度或Hessian）信息代表关键的算法支持工具。其中包括基于sciml的代理模型训练（数据驱动和/或模型通知）、参数和状态估计、模型初始化的数据同化、不确定性量化（基于hessian和梯度通知的MCMC）和定量观测系统设计。学术界和工业界的合作伙伴都参与其中，他们正在使用这些框架来开发高效的电网、个性化的精确药物计量学和改进的脑电图设计。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Understanding and quantifying parameter sensitivity of simulated systems, such as the numerical models of physical systems and mathematical renderings of neural networks, are essential in simulation-based science (SBS) and scientific machine learning (SciML). They are the key ingredients in Bayesian inference and neural network training. Seizing on the opportunity of emerging open-source Earth system model development in the Julia high-level programming language, this project is endowing these open-source models with automatic differentiation (AD) enabled derivative information, making these converging data science and simulation-based science tools available to a much broader research and data science community. Enabling a general-purpose AD framework which can handle both large-scale Earth system models as well as SciML algorithms, such as physics-informed neural networks or neural differential equations, will enable seamless integration of these approaches for hybrid Bayesian inversion and Bayesian machine learning. It merges big data science, in which available data enable model discovery with sparse data science, and the model structure is exploited in the selection of surrogate models representing data-informed subspaces and fulfilling conservation laws. The emerging Julia language engages a new generation of researchers and software engineers, channeling much needed talent into computational science approaches to climate modeling. Through dedicated community outreach programs (e.g., Hackathons, Minisymposia, Tutorials) the project team will be working toward increasing equity, diversity, and inclusion across the participating disciplines.The project is developing a framework for universal differentiable programming and open-source, general-purpose AD that unifies these algorithmic frameworks within Julia programming language. The general-purpose AD framework in Julia leverages the composability of Julia software packages and the differentiable programming approach that underlies many of the SciML and high-performance scientific computing packages. Compared to most current modeling systems targeted for HPC, Julia is ideally suited for heterogeneous parallel computing hardware (e.g., CUDA, ROCm, oneAPI, ARM, PowerPC, x86 64, TPUs). The project is bringing together expertise in AD targeted at Earth system data assimilation in high performance computing environments with SciML expertise. The project team is working with the Julia Computing organization and package developers to ensure sustainability of the developed frameworks. The project’s Earth system flagship applications consist of (i) an open-source, AD-enabled ocean general circulation model that is being developed separately as part of the Climate Modelling Alliance (CliMA), and (2) an open-source, AD-enabled ice flow model. Each of these application frameworks is being made available to the community for science application, in which derivative (gradient or Hessian) information represent key algorithmic enabling tools. These include SciML-based training of surrogate models (data-driven and/or model-informed), parameter and state estimation, data assimilation for model initialization, uncertainty quantification (Hessian-based and gradient-informed MCMC) and quantitative observing system design. Academic and industry partners are involved, who are using the frameworks for developing efficient power grids, personalized precision pharmacometrics, and improved EEG design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.