Deep Learning for Inverse Problems

逆问题的深度学习

• 批准号: 2011699
• 负责人: Lexing Ying
• 金额: $ 40万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-15 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2011699&HistoricalAwards=false
• 关键词: Deep; Learning; Inverse; Problems

In the past few years, deep learning has become the dominant approach for computer vision, image processing, speech recognition, and many other applications in machine learning and data science. This success is a synergy of several ingredients: (1) neural networks (NNs) as a flexible framework for representing high-dimensional functions and maps, (2) simple algorithms such as back-propagation and stochastic gradient descent for tuning the model parameters, (3) highly effective general software packages such as Tensorflow and Pytorch, and (4) unprecedented computing power. Despite the successes however, several key challenges remain: (1) the NN architectural design is still an art and it lacks basic mathematical principles in many cases, and (2) the NN training often requires an enormous amount of data, which is not available in many applications. Many computational problems in physical sciences also face the same challenges as those in data science: high-dimensionality, complicated or unspecified models, and large computational cost. Some well-known examples are many-body quantum systems, deterministic and stochastic control, molecular dynamics, uncertainty quantification, and inverse problems. It is natural to leverage the recent developments of NNs in the study of these problems. This project focuses on inverse problems, that is, recovering unknown problem parameters from observation data. It is a field of enormous importance, with applications in physics, chemistry, medicine, earth sciences, and defense. However, many inverse problems are known to be computationally challenging. This project will use deep learning as a means for solving these inverse problems more efficiently. The PI plans to develop a new applied linear algebra course with a machine learning focus and to organize workshops at the interface of deep learning and physical sciences. The project will train graduate and undergraduate students through the research and support one graduate student per year.As a tool for representing high-dimensional maps, neural nets (NN) offer a flexible way for representing the full inverse maps and learn the data distribution prior via training. The rich mathematical and physical theories behind inverse problems provide theoretical guidance for designing compact yet effective NN architectures and hence it is possible to avoid the need for an enormous amount of data. In the theoretical part, this project plans to identify the commonly-used mathematical operators in many inverse problems and design novel NN modules for these operators. Two such examples are the pseudodifferential operators and the Fourier integral operators. Using analytical results from partial differential equations and numerical linear algebra, both types of operators can be represented compactly and accurately as NN modules. In the application part, this project considers five inverse problems: electric impedance tomography, optical tomography, inverse acoustic/electromagnetic scattering, seismic imaging, and travel-time tomography. The NN for the inverse map is then assembled using the modules from the theory part, along with existing primitives such as convolutional NN. The whole NN is then trained end-to-end with the training data. The research results such as publications and software will be made available in public domain. The project will also lead to broader impacts in several areas. First, this project advocates for a more principled approach for NN architecture design based on mathematical theories, sparsity considerations, and invariance/equivariance principles. Second, the project covers inverse problems from five different topics. By working with scientists and engineers, the PI plans to apply the technologies developed to practical applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去的几年里，深度学习已经成为计算机视觉、图像处理、语音识别以及机器学习和数据科学中许多其他应用的主要方法。这一成功是几个因素协同作用的结果：(1)神经网络(NNS)作为表示高维函数和地图的灵活框架，(2)用于调整模型参数的简单算法，如反向传播和随机梯度下降，(3)高效的通用软件包，如TensorFlow和Pytorch，以及(4)前所未有的计算能力。然而，尽管取得了成功，几个关键的挑战仍然存在：(1)神经网络建筑设计仍然是一门艺术，在许多情况下缺乏基本的数学原理，(2)神经网络训练经常需要大量的数据，而这在许多应用中是无法获得的。物理科学中的许多计算问题也面临着与数据科学中相同的挑战：高维、复杂或未指明的模型，以及巨大的计算成本。一些著名的例子是多体量子系统、确定性和随机控制、分子动力学、不确定性量化和反问题。在研究这些问题时，利用NNS的最新发展是很自然的。这个项目的重点是反问题，即从观测数据中恢复未知问题的参数。这是一个非常重要的领域，在物理、化学、医学、地球科学和国防方面都有应用。然而，众所周知，许多反问题在计算上是具有挑战性的。这个项目将使用深度学习作为一种更有效地解决这些逆问题的方法。PI计划开发一门以机器学习为重点的新的应用线性代数课程，并在深度学习和物理科学的交汇点上组织讲习班。该项目将通过研究培养研究生和本科生，每年支持一名研究生。神经网络作为表示高维地图的工具，提供了一种灵活的方式来表示全逆地图，并通过训练学习数据的先验分布。反问题背后丰富的数学和物理理论为设计紧凑而有效的神经网络结构提供了理论指导，因此可以避免对大量数据的需要。在理论部分，本项目计划识别出许多反问题中常用的数学算子，并为这些算子设计新的神经网络模型。两个这样的例子是拟微分算子和傅立叶积分算子。利用偏微分方程组和数值线性代数的分析结果，这两类算子都可以简洁而准确地表示为神经网络模块。在应用部分，本项目考虑了五个反问题：电阻抗层析成像、光学层析成像、声波/电磁散射反问题、地震成像和旅行时层析成像。然后使用来自理论部分的模块以及现有的基元(例如卷积NN)来组装用于逆映射的NN。然后用训练数据对整个NN进行端到端的训练。出版物和软件等研究成果将在公共领域提供。该项目还将在几个领域产生更广泛的影响。首先，该项目倡导一种基于数学理论、稀疏性考虑和不变性/等方差原则的更原则性的神经网络体系结构设计方法。其次，该项目涵盖了五个不同主题的反问题。通过与科学家和工程师合作，PI计划将开发的技术应用于实际应用。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。