RI:Small:NSF-BSF: Computational and Statistical Tradeoffs in Inverse Problems using Deep Learning

RI:Small:NSF-BSF：使用深度学习的逆问题中的计算和统计权衡

• 批准号: 1816753
• 负责人: Joan Bruna Estrach
• 金额: $ 49.98万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816753&HistoricalAwards=false
• 关键词: RI; Small; NSF; BSF; Computational

In many real life scenarios one may want to measure some information about a certain object, but cannot measure directly, but only indirectly. For example, if a doctor wants to see the lungs of a patient he cannot do it by just looking at them, but needs to use a special machine such as a CT scanner. This scanner emits radiation on the patient and then measures the reflected radiation to gain information about the shape of the lungs. There are many other examples often encountered in photography, telecommunication, or navigation and more that share the same challenge of inferring signals of interest from indirect, noisy measurements: these are known as inverse problems. While many techniques exist for reconstructing the original object from its measurements, one of the main challenges is to do it in a timely manner. This research aims at providing novel computationally efficient techniques for solving inverse problems. In particular, it explores the ability of deep neural networks to accelerate and improve traditional techniques. This research focuses on both the theoretical aspects of such methods, needed for example to certify that the reconstructed image from a patient CT scan using a deep neural network is close to the underlying original scan, and their applications to tomography and seismic imaging. This research provides transformative multidisciplinary educational opportunities, developed in the Center for Data Science, New York University, bringing together signal processing, statistics and machine learning in new graduate level courses. Thanks to the wide range of relevant applications, this research also provides unique outreach activities to K12 and highschool students, as well as minority students through the BSF collaboration.Inverse problems occur in many fields ranging from signal processing to machine learning and are relevant to many domains such as medicine (e.g., getting an image in computer tomography) and physics (e.g., calculating the density of the earth from measurements of its gravity fields). Many solvers have been developed for these type of problems, leveraging specific high-dimensional statistical models for the signals of interest. However, there are three main challenges that persist in current existing solutions. First, many of them are computationally demanding and therefore not applicable to many applications that require a solution in a timely manner. Second, many inverse problems pose non-convex optimization problems that do not have an efficient solution at all. Third, current methodology do not make optimal use of available training examples, which, depending on the application, ranges from tens of instances to millions. This project provides novel theoretical foundations for the neural network based solver on general inverse problems, extends this theory to include non-linear problems that do not admit convex solutions, as well as graph-structured problems, and demonstrates its efficiency on challenging applications including low-dose Computed Tomography, Seismic spike de- convolution, and Quantum State Tomography. It specifically builds on the combination of two recent tools developed by the PI and his Israeli collaborator, which provide complimentary insights on the mechanisms underpinning the neural network acceleration. This novel theoretical framework enables a series of important extensions and generalizations, such as non-linear inverse problems, partially known measuring operators, and distributed optimization. As part of this research, several educational outreach activities are conducted to make neural networks more accessible to the broad student community, including K12, highschool and minority students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在许多真实的生活场景中，人们可能想要测量关于某个对象的一些信息，但不能直接测量，而只能间接测量。例如，如果医生想看到病人的肺部，他不能只看它们，而是需要使用特殊的机器，如CT扫描仪。该扫描仪向患者发射辐射，然后测量反射的辐射，以获得有关肺部形状的信息。在摄影、电信或导航中经常会遇到许多其他例子，这些例子都面临着从间接的噪声测量中推断感兴趣信号的挑战：这些被称为逆问题。虽然有许多技术可以根据测量结果重建原始物体，但主要挑战之一是及时完成。本研究的目的是提供新的计算效率高的技术来解决反问题。特别是，它探索了深度神经网络加速和改进传统技术的能力。本研究的重点是这些方法的理论方面，例如，需要证明使用深度神经网络从患者CT扫描重建的图像接近底层原始扫描，以及它们在断层扫描和地震成像中的应用。这项研究提供了变革性的多学科教育机会，在纽约大学数据科学中心开发，在新的研究生课程中汇集了信号处理，统计和机器学习。由于广泛的相关应用，这项研究还通过BSF合作为K12和高中学生以及少数民族学生提供了独特的外展活动。逆问题发生在从信号处理到机器学习的许多领域，并且与许多领域相关，例如医学（例如，在计算机断层摄影中获得图像）和物理学（例如，根据重力场的测量计算地球的密度）。针对这些类型的问题，已经开发了许多求解器，利用特定的高维统计模型来处理感兴趣的信号。然而，当前现有的解决方案仍然存在三个主要挑战。首先，它们中的许多都是计算要求高的，因此不适用于许多需要及时解决方案的应用程序。其次，许多反问题提出了非凸优化问题，根本没有有效的解决方案。第三，目前的方法没有最佳利用现有的训练示例，根据应用程序，范围从数十到数百万的实例。该项目为基于神经网络的一般逆问题求解器提供了新的理论基础，将该理论扩展到包括不允许凸解的非线性问题以及图结构问题，并在具有挑战性的应用中证明了其效率，包括低剂量计算机断层扫描，地震尖峰去卷积和量子状态断层扫描。它特别建立在PI和他的以色列合作者最近开发的两个工具的组合之上，这些工具提供了对神经网络加速机制的补充见解。这种新的理论框架，使一系列重要的扩展和推广，如非线性反问题，部分已知的测量算子，和分布式优化。作为这项研究的一部分，进行了几项教育推广活动，使神经网络更容易为广大的学生社区，包括K12，高中和少数民族学生。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

项目成果

Offline Contextual Bandits with Overparametrised Models

具有过度参数化模型的离线上下文强盗

• 发表时间: 2021
• 期刊: international conference on machine learning
• 作者: David, Brandfonbrener;William, F. Whitney;Rajesh, Ranganath;Joan, Bruna
• 通讯作者: Joan, Bruna

Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias

用卷积大海捞针：论架构偏差的好处

• 发表时间: 2019
• 期刊: Advances in neural information processing systems
• 作者: d'Ascoli, Stephane;Sagun, Levent;Bruna, Joan;Biroli, Giulio
• 通讯作者: Biroli, Giulio

Cartoon Explanations of Image Classifiers

• DOI: 10.1007/978-3-031-19775-8_26
• 发表时间: 2021-10
• 作者: S. Kolek;Duc Anh Nguyen;R. Levie;Joan Bruna;Gitta Kutyniok

When does return-conditioned supervised learning work for offline reinforcement learning?

返回条件监督学习何时适用于离线强化学习？

• 发表时间: 2022
• 期刊: Advances in neural information processing systems
• 作者: Brandfronbrener, David;Bietti, Alberto;Buckman, Jacob;Laroche, Romain;Bruna, Joan
• 通讯作者: Bruna, Joan

On Graph Neural Networks versus Graph-Augmented MLPs

• 发表时间: 2020-10
• 期刊: ArXiv
• 作者: Lei Chen;Zhengdao Chen;Joan Bruna