CAREER: Theoretical Foundations for Probabilistic Models with Dense Random Matrices

许多现实世界的科学和工程应用需要对大型和复杂的数据集进行复杂的处理。示例包括无线通信、计算摄影和用于分类任务的多层网络的训练。在某些情况下，性能从根本上受到数据量的限制。在其他情况下，主要限制是处理算法的计算复杂性。研究人员面临的一个主要挑战是理解这些基本限制。本项目通过研究概率模型来探索这些限制，该模型描述了数据与未知感兴趣量（例如，传输的消息或正确的标签）之间的统计关系。这项研究包括结合信息论和统计物理学的思想来计算基本极限，并利用这些结果来设计具有改进性能的有效方法。这项研究的跨学科性质反映在该项目的教育活动中，其重点是建立工程学、统计物理学和信息科学之间的联系，并通过探索性数据分析改善本科教育。这项研究背后的关键概念是，通过密集随机矩阵的乘法引起的统计依赖性可以通过与涉及加性高斯噪声的更简单模型的联系来理解。在最近的一项突破中，研究人员展示了信息理论的思想如何为使用统计物理学的启发式复制方法推测的行为提供严格的证据。基于这一见解，研究围绕三个重点进行：i)发展新的理论方法，以提供对基本极限的严格和可解释的表征；ii)为推理、学习和压缩设计新的算法；iii)分析双线性和多层推理问题及其在深度学习中的应用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Many real-world scientific and engineering applications require sophisticated processing of large and complex data sets. Examples include wireless communications, computational photography, and the training of multilayer networks for classification tasks. In some cases, performance is fundamentally limited by the amount of data. In other cases, the main limitation is the computational complexity of processing algorithms. A major challenge for researchers is to understand these fundamental limits. This project explores these limits by studying probabilistic models that describe the statistical relationship between the data and the unknown quantities of interest (e.g., transmitted message or correct label). The research involves combining ideas from information theory and statistical physics to compute fundamental limits and using these results to design efficient methods with improved performance. The interdisciplinary nature of the research is mirrored in the education activities of this project, which focuses on making connections between engineering, statistical physics, and the information sciences, as well as improving undergraduate education through exploratory data analysis.The key conceptual idea behind this research is that statistical dependencies induced through multiplication by dense random matrices can be understood through connections with simpler models involving additive Gaussian noise. In a recent breakthrough, the investigator showed how ideas from information theory could provide rigorous proofs for behaviors that had been conjectured using the heuristic replica method from statistical physics. Building upon this insight, the research is organized around three thrusts: i) Developing new theoretical methods to provide rigorous and interpretable characterization of fundamental limits; ii) Designing new algorithms for inference, learning, and compression; and iii) Analyzing bi-linear and multi-layer inference problems with applications to deep learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.