Collaborative Research: CIF: Medium: Learning and Inference in High-Dimensional Models: Rigorous Analysis and Applications

当代信号处理和机器学习问题的一个关键特征是它们的大规模。信号处理任务通常涉及数百万像素的图像和视频，而现代深度学习方法通常涉及数百万可调参数。尽管最近的方法，特别是深度学习，在高维环境中取得了巨大的实践成功，但它们很难从理论的角度来解释。该项目旨在开发数学工具，以便沿着以下方向更好地理解此类估计和学习问题：如何可追溯地制定对当代问题的精确，高维分析；这些分析说明了关于估计和学习的信息论限制；如何用实用的算法来接近这些极限呢？为了产生更广泛的影响，该项目包括在研讨会上进行传播，与不断发展的机器学习行业进行协调，并开发一个关于数据科学的新模块，向高中教师提供。该项目建立在强大的近似消息传递（AMP）框架之上，这是一种评估方法，为对现代高维问题的严格分析理解提供了潜力。自从AMP作为一种理解压缩感知中线性逆问题的方法出现以来，它在广泛的估计和学习任务中取得了巨大的成功。该项目旨在将AMP框架扩展到当代的大规模学习任务中。该项目分为三个主要部分：1)结构化双线性模型的推理，2)多层神经网络的学习，以及3)傅里叶和卷积算子的分析。在每个推力中，该项目将发展基本的数学理论并验证关键应用的理论，特别是在图像处理和统计学习方面。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A key feature of contemporary signal processing and machine learning problems is their massive scale. Signal processing tasks routinely involve images and videos with millions of pixels, and modern deep-learning methods often involve millions of tunable parameters. Although recent methods, particularly deep learning, have had tremendous practical success in the high-dimensional setting, they are difficult to explain from a theoretical perspective. This project seeks to develop mathematical tools to better understand such estimation and learning problems along the following directions: How does one tractably formulate precise, high-dimensional analyses of contemporary problems; what do those analyses say about the information-theoretic limits of estimation and learning; and how can these limits be approached by practical algorithms? To achieve broader impacts, the project includes dissemination in workshops, coordination with the growing machine learning industry and the development of a new module on data science to be provided to high school teachers.The project builds on the powerful approximate message passing (AMP) framework, an estimation methodology that offers the potential for a rigorous analytic understanding of modern, high-dimensional problems. Since its origin as a method for understanding linear inverse problems in compressed sensing, AMP has had tremendous success in a wide range of estimation and learning tasks. This project aims to extend the AMP framework to contemporary, large-scale learning tasks. The project is organized into three main thrusts: 1) inference with structured bilinear models, 2) learning of multi-layer neural networks, and 3) analysis with Fourier and convolutional operators. In each thrust, the project will develop fundamental mathematical theory and validate the theory on key applications, particular in image processing and statistical learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.