Collaborative Research: Frameworks: Convergence of Bayesian inverse methods and scientific machine learning in Earth system models through universal differentiable programming

理解和量化模拟系统的参数敏感性，例如物理系统的数值模型和神经网络的数学渲染，在基于模拟的科学（SBS）和科学机器学习（SciML）中至关重要。它们是贝叶斯推理和神经网络训练的关键成分。该项目抓住了Julia高级编程语言中新兴的开源地球系统模型开发的机会，为这些开源模型赋予了自动微分（AD）功能的衍生信息，使这些融合的数据科学和基于模拟的科学工具可用于更广泛的研究和数据科学社区。启用通用AD框架，可以处理大规模地球系统模型以及SciML算法，例如物理信息神经网络或神经微分方程，将使这些方法无缝集成，用于混合贝叶斯反演和贝叶斯机器学习。它融合了大数据科学，其中可用的数据使模型发现与稀疏数据科学相结合，模型结构被用于选择代表数据信息子空间和满足守恒定律的代理模型。新兴的Julia语言吸引了新一代的研究人员和软件工程师，将急需的人才引入气候建模的计算科学方法。通过专门的社区外展计划（例如，Hackathons，Minisymopsia，Tuesday），项目团队将致力于提高参与学科的公平性，多样性和包容性。该项目正在开发一个通用可区分编程和开源通用AD的框架，该框架将这些算法框架统一在Julia编程语言中。Julia中的通用AD框架利用了Julia软件包的可组合性以及作为许多SciML和高性能科学计算包基础的可区分编程方法。与目前大多数针对HPC的建模系统相比，Julia非常适合异构并行计算硬件（例如，CUDA、ROCm、oneAPI、ARM、PowerPC、x86 64、TPU）。该项目汇集了AD的专业知识，目标是在高性能计算环境中与SciML专业知识进行地球系统数据同化。项目团队正在与Julia Computing组织和软件包开发人员合作，以确保所开发框架的可持续性。该项目的地球系统旗舰应用程序包括（i）一个开源的、支持AD的海洋环流模型，该模型作为气候建模联盟（Clima）的一部分单独开发，以及（2）一个开源的、支持AD的冰流模型。这些应用框架中的每一个都可供社区用于科学应用，其中导数（梯度或Hessian）信息代表关键的算法使能工具。这些包括基于SciML的代理模型训练（数据驱动和/或模型通知），参数和状态估计，模型初始化的数据同化，不确定性量化（基于Hessian和梯度通知MCMC）和定量观测系统设计。学术界和工业界的合作伙伴参与其中，他们正在使用该框架开发高效的电网、个性化的精确药物计量学和改进的EEG设计。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Understanding and quantifying parameter sensitivity of simulated systems, such as the numerical models of physical systems and mathematical renderings of neural networks, are essential in simulation-based science (SBS) and scientific machine learning (SciML). They are the key ingredients in Bayesian inference and neural network training. Seizing on the opportunity of emerging open-source Earth system model development in the Julia high-level programming language, this project is endowing these open-source models with automatic differentiation (AD) enabled derivative information, making these converging data science and simulation-based science tools available to a much broader research and data science community. Enabling a general-purpose AD framework which can handle both large-scale Earth system models as well as SciML algorithms, such as physics-informed neural networks or neural differential equations, will enable seamless integration of these approaches for hybrid Bayesian inversion and Bayesian machine learning. It merges big data science, in which available data enable model discovery with sparse data science, and the model structure is exploited in the selection of surrogate models representing data-informed subspaces and fulfilling conservation laws. The emerging Julia language engages a new generation of researchers and software engineers, channeling much needed talent into computational science approaches to climate modeling. Through dedicated community outreach programs (e.g., Hackathons, Minisymposia, Tutorials) the project team will be working toward increasing equity, diversity, and inclusion across the participating disciplines.The project is developing a framework for universal differentiable programming and open-source, general-purpose AD that unifies these algorithmic frameworks within Julia programming language. The general-purpose AD framework in Julia leverages the composability of Julia software packages and the differentiable programming approach that underlies many of the SciML and high-performance scientific computing packages. Compared to most current modeling systems targeted for HPC, Julia is ideally suited for heterogeneous parallel computing hardware (e.g., CUDA, ROCm, oneAPI, ARM, PowerPC, x86 64, TPUs). The project is bringing together expertise in AD targeted at Earth system data assimilation in high performance computing environments with SciML expertise. The project team is working with the Julia Computing organization and package developers to ensure sustainability of the developed frameworks. The project’s Earth system flagship applications consist of (i) an open-source, AD-enabled ocean general circulation model that is being developed separately as part of the Climate Modelling Alliance (CliMA), and (2) an open-source, AD-enabled ice flow model. Each of these application frameworks is being made available to the community for science application, in which derivative (gradient or Hessian) information represent key algorithmic enabling tools. These include SciML-based training of surrogate models (data-driven and/or model-informed), parameter and state estimation, data assimilation for model initialization, uncertainty quantification (Hessian-based and gradient-informed MCMC) and quantitative observing system design. Academic and industry partners are involved, who are using the frameworks for developing efficient power grids, personalized precision pharmacometrics, and improved EEG design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.