Collaborative Research: CIF: Medium: Learning and Inference in High-Dimensional Models: Rigorous Analysis and Applications

当代信号处理和机器学习问题的一个关键特征是它们的规模巨大。信号处理任务通常涉及数百万像素的图像和视频，而现代深度学习方法通常涉及数百万个可调参数。虽然最近的方法，特别是深度学习，在高维环境下取得了巨大的实践成功，但它们很难从理论的角度进行解释。这个项目寻求开发数学工具，以更好地理解这样的估计和学习问题，沿着以下方向：一个人如何轻松地对当代问题进行精确的、高维度的分析；这些分析对估计和学习的信息论限制说了什么；以及如何通过实际算法接近这些限制？为了实现更广泛的影响，该项目包括在研讨会上进行传播，与不断增长的机器学习行业协调，以及开发一个新的数据科学模块，提供给高中教师。该项目建立在强大的近似消息传递(AMP)框架的基础上，这是一种估计方法，为严格分析理解现代高维问题提供了可能性。自从AMP作为一种理解压缩感知中线性逆问题的方法以来，它在广泛的估计和学习任务中取得了巨大的成功。该项目旨在将AMP框架扩展到当代的大规模学习任务。该项目被组织成三个主要推动力：1)使用结构化双线性模型进行推理，2)学习多层神经网络，以及3)使用傅立叶和卷积算子进行分析。在每一个推力中，该项目都将发展基本的数学理论，并在关键应用方面验证该理论，特别是在图像处理和统计学习方面。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

A key feature of contemporary signal processing and machine learning problems is their massive scale. Signal processing tasks routinely involve images and videos with millions of pixels, and modern deep-learning methods often involve millions of tunable parameters. Although recent methods, particularly deep learning, have had tremendous practical success in the high-dimensional setting, they are difficult to explain from a theoretical perspective. This project seeks to develop mathematical tools to better understand such estimation and learning problems along the following directions: How does one tractably formulate precise, high-dimensional analyses of contemporary problems; what do those analyses say about the information-theoretic limits of estimation and learning; and how can these limits be approached by practical algorithms? To achieve broader impacts, the project includes dissemination in workshops, coordination with the growing machine learning industry and the development of a new module on data science to be provided to high school teachers.The project builds on the powerful approximate message passing (AMP) framework, an estimation methodology that offers the potential for a rigorous analytic understanding of modern, high-dimensional problems. Since its origin as a method for understanding linear inverse problems in compressed sensing, AMP has had tremendous success in a wide range of estimation and learning tasks. This project aims to extend the AMP framework to contemporary, large-scale learning tasks. The project is organized into three main thrusts: 1) inference with structured bilinear models, 2) learning of multi-layer neural networks, and 3) analysis with Fourier and convolutional operators. In each thrust, the project will develop fundamental mathematical theory and validate the theory on key applications, particular in image processing and statistical learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.