Collaborative Research: Frameworks: Convergence of Bayesian inverse methods and scientific machine learning in Earth system models through universal differentiable programming

在基于模拟的科学(SBS)和科学机器学习(SciML)中，理解和量化模拟系统的参数敏感度，如物理系统的数值模型和神经网络的数学表示，是必不可少的。它们是贝叶斯推理和神经网络训练的关键成分。该项目抓住了用Julia高级编程语言开发新兴的开源地球系统模型的机会，赋予这些开源模型以自动微分(AD)功能的派生信息，使这些汇聚数据科学和基于模拟的科学工具可供更广泛的研究和数据科学界使用。启用既可以处理大规模地球系统模型又可以处理SciML算法的通用AD框架，例如物理信息神经网络或神经微分方程，将使这些方法能够无缝集成，以实现混合贝叶斯反演和贝叶斯机器学习。它融合了大数据科学和稀疏数据科学，其中可用数据使模型发现成为可能，模型结构被用于选择代表数据信息子空间和满足守恒定律的代理模型。新兴的Julia语言吸引了新一代研究人员和软件工程师，将急需的人才引入到气候建模的计算科学方法中。通过专门的社区推广计划(例如，黑客马拉松、迷你符号、教程)，项目团队将致力于提高参与学科的公平性、多样性和包容性。该项目正在开发一个框架，用于通用可区分编程和开源、通用AD，在Julia编程语言中统一这些算法框架。Julia中的通用AD框架利用了Julia软件包的可组合性，以及作为许多SciML和高性能科学计算软件包基础的可区分编程方法。与目前大多数面向高性能计算的建模系统相比，Julia非常适合于异构并行计算硬件(例如，CUDA、ROCM、One API、ARM、PowerPC、x86、TPU)。该项目将以高性能计算环境中的地球系统数据同化为目标的AD方面的专业知识与SciML的专业知识结合起来。项目团队正在与Julia计算组织和包开发人员合作，以确保所开发框架的可持续性。该项目的地球系统旗舰应用包括：(1)作为气候模拟联盟(CLMA)的一部分单独开发的开放源码的启用AD的海洋环流模型，以及(2)开放源码的启用AD的冰流模型。这些应用程序框架中的每一个都提供给社区用于科学应用，其中派生(梯度或黑森)信息代表关键的算法使能工具。其中包括以本科学知识为基础的替代模型培训(数据驱动和/或模型知情)、参数和状态估计、用于模型初始化的数据同化、不确定性量化(基于海森和梯度信息的MCMC)以及定量观测系统设计。学术和行业合作伙伴参与其中，他们正在使用框架开发高效电网、个性化精确药物计量学和改进的脑电设计。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Understanding and quantifying parameter sensitivity of simulated systems, such as the numerical models of physical systems and mathematical renderings of neural networks, are essential in simulation-based science (SBS) and scientific machine learning (SciML). They are the key ingredients in Bayesian inference and neural network training. Seizing on the opportunity of emerging open-source Earth system model development in the Julia high-level programming language, this project is endowing these open-source models with automatic differentiation (AD) enabled derivative information, making these converging data science and simulation-based science tools available to a much broader research and data science community. Enabling a general-purpose AD framework which can handle both large-scale Earth system models as well as SciML algorithms, such as physics-informed neural networks or neural differential equations, will enable seamless integration of these approaches for hybrid Bayesian inversion and Bayesian machine learning. It merges big data science, in which available data enable model discovery with sparse data science, and the model structure is exploited in the selection of surrogate models representing data-informed subspaces and fulfilling conservation laws. The emerging Julia language engages a new generation of researchers and software engineers, channeling much needed talent into computational science approaches to climate modeling. Through dedicated community outreach programs (e.g., Hackathons, Minisymposia, Tutorials) the project team will be working toward increasing equity, diversity, and inclusion across the participating disciplines.The project is developing a framework for universal differentiable programming and open-source, general-purpose AD that unifies these algorithmic frameworks within Julia programming language. The general-purpose AD framework in Julia leverages the composability of Julia software packages and the differentiable programming approach that underlies many of the SciML and high-performance scientific computing packages. Compared to most current modeling systems targeted for HPC, Julia is ideally suited for heterogeneous parallel computing hardware (e.g., CUDA, ROCm, oneAPI, ARM, PowerPC, x86 64, TPUs). The project is bringing together expertise in AD targeted at Earth system data assimilation in high performance computing environments with SciML expertise. The project team is working with the Julia Computing organization and package developers to ensure sustainability of the developed frameworks. The project’s Earth system flagship applications consist of (i) an open-source, AD-enabled ocean general circulation model that is being developed separately as part of the Climate Modelling Alliance (CliMA), and (2) an open-source, AD-enabled ice flow model. Each of these application frameworks is being made available to the community for science application, in which derivative (gradient or Hessian) information represent key algorithmic enabling tools. These include SciML-based training of surrogate models (data-driven and/or model-informed), parameter and state estimation, data assimilation for model initialization, uncertainty quantification (Hessian-based and gradient-informed MCMC) and quantitative observing system design. Academic and industry partners are involved, who are using the frameworks for developing efficient power grids, personalized precision pharmacometrics, and improved EEG design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.