RI:Small:NSF-BSF: Computational and Statistical Tradeoffs in Inverse Problems using Deep Learning

RI:Small:NSF-BSF：使用深度学习的逆问题中的计算和统计权衡

• 批准号: 1816753
• 负责人: Joan Bruna Estrach
• 金额: $ 49.98万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816753&HistoricalAwards=false
• 关键词: RI; Small; NSF; BSF; Computational

In many real life scenarios one may want to measure some information about a certain object, but cannot measure directly, but only indirectly. For example, if a doctor wants to see the lungs of a patient he cannot do it by just looking at them, but needs to use a special machine such as a CT scanner. This scanner emits radiation on the patient and then measures the reflected radiation to gain information about the shape of the lungs. There are many other examples often encountered in photography, telecommunication, or navigation and more that share the same challenge of inferring signals of interest from indirect, noisy measurements: these are known as inverse problems. While many techniques exist for reconstructing the original object from its measurements, one of the main challenges is to do it in a timely manner. This research aims at providing novel computationally efficient techniques for solving inverse problems. In particular, it explores the ability of deep neural networks to accelerate and improve traditional techniques. This research focuses on both the theoretical aspects of such methods, needed for example to certify that the reconstructed image from a patient CT scan using a deep neural network is close to the underlying original scan, and their applications to tomography and seismic imaging. This research provides transformative multidisciplinary educational opportunities, developed in the Center for Data Science, New York University, bringing together signal processing, statistics and machine learning in new graduate level courses. Thanks to the wide range of relevant applications, this research also provides unique outreach activities to K12 and highschool students, as well as minority students through the BSF collaboration.Inverse problems occur in many fields ranging from signal processing to machine learning and are relevant to many domains such as medicine (e.g., getting an image in computer tomography) and physics (e.g., calculating the density of the earth from measurements of its gravity fields). Many solvers have been developed for these type of problems, leveraging specific high-dimensional statistical models for the signals of interest. However, there are three main challenges that persist in current existing solutions. First, many of them are computationally demanding and therefore not applicable to many applications that require a solution in a timely manner. Second, many inverse problems pose non-convex optimization problems that do not have an efficient solution at all. Third, current methodology do not make optimal use of available training examples, which, depending on the application, ranges from tens of instances to millions. This project provides novel theoretical foundations for the neural network based solver on general inverse problems, extends this theory to include non-linear problems that do not admit convex solutions, as well as graph-structured problems, and demonstrates its efficiency on challenging applications including low-dose Computed Tomography, Seismic spike de- convolution, and Quantum State Tomography. It specifically builds on the combination of two recent tools developed by the PI and his Israeli collaborator, which provide complimentary insights on the mechanisms underpinning the neural network acceleration. This novel theoretical framework enables a series of important extensions and generalizations, such as non-linear inverse problems, partially known measuring operators, and distributed optimization. As part of this research, several educational outreach activities are conducted to make neural networks more accessible to the broad student community, including K12, highschool and minority students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在许多现实生活场景中，人们可能想要测量某个对象的一些信息，但无法直接测量，而只能间接测量。例如，如果医生想要看到病人的肺部，他不能仅仅通过观察来做到这一点，而是需要使用诸如CT扫描仪之类的特殊机器。该扫描仪向患者发射辐射，然后测量反射的辐射以获得有关肺部形状的信息。在摄影、电信或导航等领域经常遇到的许多其他例子都面临着从间接、噪声测量中推断感兴趣信号的相同挑战：这些被称为逆问题。虽然存在许多根据测量结果重建原始物体的技术，但主要挑战之一是及时完成。这项研究旨在提供新颖的计算有效技术来解决反问题。特别是，它探索了深度神经网络加速和改进传统技术的能力。这项研究重点关注此类方法的理论方面，例如需要证明使用深度神经网络从患者 CT 扫描中重建的图像接近底层原始扫描，以及它们在断层扫描和地震成像中的应用。这项研究提供了变革性的多学科教育机会，由纽约大学数据科学中心开发，将信号处理、统计学和机器学习整合到新的研究生水平课程中。由于相关应用范围广泛，这项研究还通过 BSF 合作为 K12 和高中生以及少数族裔学生提供了独特的推广活动。逆问题发生在从信号处理到机器学习的许多领域，并且与许多领域相关，例如医学（例如，在计算机断层扫描中获取图像）和物理学（例如，通过测量地球的重力场来计算地球的密度）。针对此类问题开发了许多求解器，利用特定的高维统计模型来处理感兴趣的信号。然而，当前现有解决方案仍然存在三个主要挑战。首先，其中许多计算要求较高，因此不适用于许多需要及时解决的应用程序。其次，许多逆问题提出了根本没有有效解决方案的非凸优化问题。第三，当前的方法没有充分利用可用的训练示例，根据应用的不同，训练示例的数量从数十个到数百万个不等。该项目为基于神经网络的一般逆问题求解器提供了新颖的理论基础，将该理论扩展到包括不允许凸解的非线性问题以及图结构问题，并证明了其在低剂量计算机断层扫描、地震尖峰反卷积和量子态断层扫描等具有挑战性的应用中的效率。它特别建立在 PI 和他的以色列合作者最近开发的两个工具的组合之上，这两个工具提供了关于支持神经网络加速的机制的补充见解。这种新颖的理论框架实现了一系列重要的扩展和概括，例如非线性反问题、部分已知的测量算子和分布式优化。作为这项研究的一部分，开展了多项教育推广活动，以使神经网络更容易为广大学生群体所使用，包括 K12、高中和少数族裔学生。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Offline Contextual Bandits with Overparametrised Models

具有过度参数化模型的离线上下文强盗

• 发表时间: 2021
• 期刊: international conference on machine learning
• 作者: David, Brandfonbrener;William, F. Whitney;Rajesh, Ranganath;Joan, Bruna
• 通讯作者: Joan, Bruna

Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias

用卷积大海捞针：论架构偏差的好处

• 发表时间: 2019
• 期刊: Advances in neural information processing systems
• 作者: d'Ascoli, Stephane;Sagun, Levent;Bruna, Joan;Biroli, Giulio
• 通讯作者: Biroli, Giulio

Cartoon Explanations of Image Classifiers

• DOI: 10.1007/978-3-031-19775-8_26
• 发表时间: 2021-10
• 作者: S. Kolek;Duc Anh Nguyen;R. Levie;Joan Bruna;Gitta Kutyniok

When does return-conditioned supervised learning work for offline reinforcement learning?

返回条件监督学习何时适用于离线强化学习？

• 发表时间: 2022
• 期刊: Advances in neural information processing systems
• 作者: Brandfronbrener, David;Bietti, Alberto;Buckman, Jacob;Laroche, Romain;Bruna, Joan
• 通讯作者: Bruna, Joan

On Graph Neural Networks versus Graph-Augmented MLPs

• 发表时间: 2020-10
• 期刊: ArXiv
• 作者: Lei Chen;Zhengdao Chen;Joan Bruna