FRG: Collaborative Research: Variationally Stable Neural Networks for Simulation, Learning, and Experimental Design of Complex Physical Systems

在科学和技术中，一项无处不在且往往至关重要的任务是从控制物理定律和嘈杂的观测数据（如传感器系统提供的数据）中综合信息，以优化重要的感兴趣量。本项目涉及的例子包括通过多孔介质的地下流动、光纤、波导设计和材料科学应用。这个项目的首要目标是开发一个数学上严格的框架，用于从给定的信息源中“学习”潜在的复杂模型。现代机器学习（尤其是深度学习）最近在容错应用中取得了惊人的成功，但这并不意味着它在对错误敏感的科学任务中也取得了成功。针对后者，该项目旨在通过严格的精度量化和认证来显著提高预测能力，这可以说是下一代科学技术模拟工具不可或缺的特征。这需要整合来自不同领域的概念工具，如数值和函数分析、机器学习、统计学、优化和信息几何。该项目为此目的聚集了一个多元化的团队，作为副产品，为学生和年轻研究人员创造了一个独特的教育框架。根据不同的应用，控制物理定律是用不同类型的参数相关偏微分方程（PDEs）来表述的。然后，部分观察到的感兴趣的状态在（或接近）遍历参数空间时获得的所有解中。学习或优化这种状态归结为涉及许多（参数）变量函数的不适定逆问题。为了应对这些障碍，该项目制定了一个“学习”框架，作为由深度神经网络组成的假设类的非线性回归问题。残差型损失函数避免了大量高保真训练样本的昂贵计算。然后，所谓的变差正确剩余风险保证了准确性量化和后验认证。这意味着在优化的任何阶段损失的大小与在物理相关度量中产生的结果估计所产生的误差一致成正比。变分正确性是通过底层偏微分方程的稳定变分公式实现的。它们通常基于当前不断发展的（不连续的）Petrov-Galerkin方法。由于对偶模的固有出现，这需要新的策略来有效地评估在高维参数环境下产生的损失函数。此外，特别适应的梯度流将成为开发鲁棒集成优化/适应/正则化策略的重要组成部分。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A ubiquitous and often critical task in science and technology is to synthesize information from governing physical laws and noisy observational data, such as those provided by sensor systems, in order to optimize important quantities of interest. Examples touched upon in this project include subsurface flow through porous media, fiber optics, waveguide design, and material science applications. The overarching goal of this project is to develop a mathematically rigorous framework for “learning” the underlying complex models from the given sources of information. The recent stunning successes of modern machine learning, especially deep learning, in error-tolerant applications, does not automatically imply its success in error-sensitive scientific tasks. Targeting the latter, this project aims to significantly advance prediction capabilities through rigorous accuracy quantification and certification, arguably an indispensable feature of next generation simulation tools in science and technology. This requires integrating conceptual tools from diverse areas such as numerical and functional analysis, machine learning, statistics, optimization, and information geometry. The project gathers a diverse team for this purpose, and as a byproduct, creates a unique educational framework for students and young researchers. Governing physical laws are formulated in terms of (systems of) parameter dependent partial differential equations (PDEs) of various types depending on the application. Partially observed states of interest are then among (or close to) all those solutions that are obtained when traversing the parameter space. Learning or optimizing such states boils down to ill-posed inverse problems involving functions of many (parametric) variables. To cope with these obstructions, this project formulates a “learning” framework as a nonlinear regression problem over hypothesis classes comprised of deep neural networks. Residual type loss functions are employed to avoid expensive computation of a large number of high-fidelity training samples. Accuracy quantification and a posteriori certification is then warranted by so-called variationally-correct residual risks. This means that the size of the loss at any stage of the optimization is uniformly proportional to the error incurred by the resulting estimation in a physically relevant metric. The variational correctness is achieved through stable variational formulations of the underlying PDEs. They are typically based on currently evolving (discontinuous) Petrov-Galerkin methodologies. Due to the inherent appearance of dual norms, this requires new strategies for efficiently evaluating resulting loss functions in the high dimensional parametric context. Moreover, specially adapted gradient flows will serve as an important constituent in developing robust integrated optimization/adaptation/regularization strategies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.