FRG: Collaborative Research: Variationally Stable Neural Networks for Simulation, Learning, and Experimental Design of Complex Physical Systems

科学和技术中一项普遍存在且往往至关重要的任务是综合来自支配性物理定律和噪声观测数据（如传感器系统提供的数据）的信息，以优化重要的感兴趣量。在这个项目中涉及的例子包括地下流动通过多孔介质，光纤，波导设计，材料科学的应用。 该项目的首要目标是开发一个数学上严格的框架，用于从给定的信息源中“学习”底层的复杂模型。现代机器学习，特别是深度学习，在容错应用中取得的惊人成功，并不意味着它在错误敏感的科学任务中取得了成功。针对后者，该项目旨在通过严格的准确性量化和认证来显着提高预测能力，这可以说是科学和技术中下一代模拟工具不可或缺的功能。这需要集成来自不同领域的概念工具，如数值和函数分析，机器学习，统计，优化和信息几何。该项目为此目的聚集了一个多元化的团队，并作为副产品，为学生和年轻研究人员创建了一个独特的教育框架。管理物理定律制定的（系统）参数相关的偏微分方程（PDE）的各种类型取决于应用。 然后，部分观察到的感兴趣的状态是在遍历参数空间时获得的所有那些解中（或接近于这些解）。 学习或优化这些状态归结为涉及许多（参数）变量的函数的不适定逆问题。为了科普这些障碍，该项目制定了一个“学习”框架，作为由深度神经网络组成的假设类的非线性回归问题。采用残差型损失函数避免了大量高保真训练样本的昂贵计算。准确性量化和后验认证，然后保证所谓的变异正确的剩余风险。这意味着，在优化的任何阶段，损失的大小都与物理相关度量中的结果估计所引起的误差成正比。 变分的正确性是通过底层偏微分方程的稳定变分公式来实现的。它们通常基于目前不断发展的（不连续的）彼得罗夫-伽辽金方法。由于对偶范数的固有外观，这需要新的策略来有效地评估高维参数背景下的损失函数。此外，特别适应梯度流将作为一个重要的组成部分，在发展强大的综合优化/适应/regularizationstrategies.This奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

A ubiquitous and often critical task in science and technology is to synthesize information from governing physical laws and noisy observational data, such as those provided by sensor systems, in order to optimize important quantities of interest. Examples touched upon in this project include subsurface flow through porous media, fiber optics, waveguide design, and material science applications. The overarching goal of this project is to develop a mathematically rigorous framework for “learning” the underlying complex models from the given sources of information. The recent stunning successes of modern machine learning, especially deep learning, in error-tolerant applications, does not automatically imply its success in error-sensitive scientific tasks. Targeting the latter, this project aims to significantly advance prediction capabilities through rigorous accuracy quantification and certification, arguably an indispensable feature of next generation simulation tools in science and technology. This requires integrating conceptual tools from diverse areas such as numerical and functional analysis, machine learning, statistics, optimization, and information geometry. The project gathers a diverse team for this purpose, and as a byproduct, creates a unique educational framework for students and young researchers. Governing physical laws are formulated in terms of (systems of) parameter dependent partial differential equations (PDEs) of various types depending on the application. Partially observed states of interest are then among (or close to) all those solutions that are obtained when traversing the parameter space. Learning or optimizing such states boils down to ill-posed inverse problems involving functions of many (parametric) variables. To cope with these obstructions, this project formulates a “learning” framework as a nonlinear regression problem over hypothesis classes comprised of deep neural networks. Residual type loss functions are employed to avoid expensive computation of a large number of high-fidelity training samples. Accuracy quantification and a posteriori certification is then warranted by so-called variationally-correct residual risks. This means that the size of the loss at any stage of the optimization is uniformly proportional to the error incurred by the resulting estimation in a physically relevant metric. The variational correctness is achieved through stable variational formulations of the underlying PDEs. They are typically based on currently evolving (discontinuous) Petrov-Galerkin methodologies. Due to the inherent appearance of dual norms, this requires new strategies for efficiently evaluating resulting loss functions in the high dimensional parametric context. Moreover, specially adapted gradient flows will serve as an important constituent in developing robust integrated optimization/adaptation/regularization strategies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.