Collaborative Research: CIF: Medium: Learning and Inference in High-Dimensional Models: Rigorous Analysis and Applications

当代信号处理和机器学习问题的一个关键特征是它们的大规模。信号处理任务通常涉及数百万像素的图像和视频，而现代深度学习方法通常涉及数百万个可调参数。虽然最近的方法，特别是深度学习，在高维环境中取得了巨大的实际成功，但从理论角度很难解释。这个项目旨在开发数学工具，以更好地理解这样的估计和学习问题，沿着以下方向：一个人如何巧妙地制定精确的，当代问题的高维分析，这些分析说什么估计和学习的信息理论的限制，以及如何才能接近这些限制的实际算法？为了实现更广泛的影响，该项目包括在研讨会上传播，与不断发展的机器学习行业协调，以及开发一个新的数据科学模块，提供给高中教师。该项目建立在强大的近似消息传递（AMP）框架之上，这是一种估计方法，可以对现代高维问题进行严格的分析理解。由于其起源作为压缩感知中的线性逆问题的理解方法，AMP在广泛的估计和学习任务中取得了巨大的成功。该项目旨在将AMP框架扩展到当代的大规模学习任务。该项目分为三个主要方面：1）结构化双线性模型的推理，2）多层神经网络的学习，以及3）傅立叶和卷积运算符的分析。 在每一个方向上，该项目将发展基础数学理论，并在关键应用中验证理论，特别是在图像处理和统计学习方面。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A key feature of contemporary signal processing and machine learning problems is their massive scale. Signal processing tasks routinely involve images and videos with millions of pixels, and modern deep-learning methods often involve millions of tunable parameters. Although recent methods, particularly deep learning, have had tremendous practical success in the high-dimensional setting, they are difficult to explain from a theoretical perspective. This project seeks to develop mathematical tools to better understand such estimation and learning problems along the following directions: How does one tractably formulate precise, high-dimensional analyses of contemporary problems; what do those analyses say about the information-theoretic limits of estimation and learning; and how can these limits be approached by practical algorithms? To achieve broader impacts, the project includes dissemination in workshops, coordination with the growing machine learning industry and the development of a new module on data science to be provided to high school teachers.The project builds on the powerful approximate message passing (AMP) framework, an estimation methodology that offers the potential for a rigorous analytic understanding of modern, high-dimensional problems. Since its origin as a method for understanding linear inverse problems in compressed sensing, AMP has had tremendous success in a wide range of estimation and learning tasks. This project aims to extend the AMP framework to contemporary, large-scale learning tasks. The project is organized into three main thrusts: 1) inference with structured bilinear models, 2) learning of multi-layer neural networks, and 3) analysis with Fourier and convolutional operators. In each thrust, the project will develop fundamental mathematical theory and validate the theory on key applications, particular in image processing and statistical learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.