CAREER: Fast and Accurate Statistical Learning and Inference from Large-Scale Data: Theory, Methods, and Algorithms

该项目将开发用于分析大型数据集的统计方法。如此庞大的数据集正在成为科学、工程和商业许多领域的重要挑战。该研究将采用多管齐下的方法来解决这些数据集分析中的几个基本问题，重点关注三个关键领域。第一个是草图和随机投影，这是一种强大的随机数据分析方法，当数据必须在一台机器上分析时使用。第二个领域是分布式统计学习和推理，其中数据集分布在多个位置，它们之间的通信有限。第三是模型再训练，在原始训练集中添加或删除数据后，必须有效地更新统计或机器学习模型。此外，该项目将具有重要的教育组成部分，PI将开发一门关于统计机器学习的新课程。这个项目还将培养一名研究生。PI致力于在项目的各个方面实现多样性和包容性，包括妇女和代表性不足的少数民族。为该项目开发的方法将作为软件免费提供，这将允许其他人直接使用并从中受益。在速写方面，该项目将利用渐近随机矩阵理论和自由概率的强大工具来分析基本问题，如回归和聚类。在分布式学习领域，PI计划通过基于梯度的优化开发和分析分布式学习的统计方法。对于模型再训练，PI旨在研究再训练与保形预测之间的联系，以开发改进的和广泛适用的预测推理方法。在技术层面上，这项工作将涉及来自概率论的先进工具，如随机矩阵理论，以及来自数值优化的工具。通过仔细分析大规模统计分析的计算方面，这项工作将旨在弥合统计和计算观点之间的差距。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project will develop statistical methods for analyzing large datasets. Such massive datasets are emerging as an important challenge in many areas of science, engineering, and business. The research will pursue a multi-pronged approach to addressing several fundamental questions in the analysis of such datasets, focusing on three key areas. The first one is sketching and random projections, which is a powerful randomized approach to data analysis used when the data must be analyzed on a single machine. The second area is distributed statistical learning and inference, where datasets are spread across multiple locations, with limited communication among them. The third is model retraining, where statistical or machine learning models must be updated efficiently after data has been added or deleted from the original training set. In addition, the project will have a significant educational component, with the PI developing a new course on statistical machine learning. This project will also train a graduate student. The PI is committed to diversity and inclusion, including women and underrepresented minorities in all aspects of the project. The methods developed for the project will be made freely available as software, which will allow others to directly use and benefit from the results.In the area of sketching, the project will leverage powerful tools from asymptotic random matrix theory and free probability to analyze fundamental problems, such as regression and clustering. In the area of distributed learning, the PI plans to develop and analyze statistical methods for distributed learning via gradient based optimization. For model retraining, the PI aims to study the connections between retraining and conformal prediction, with the goal of developing improved and broadly applicable methods for predictive inference. On a technical level, the work will involve advanced tools from probability theory, such as random matrix theory, as well as tools from numerical optimization. By carefully analyzing computational aspects of large-scale statistical analysis, the work will aim to bridge gaps between the statistical and computational perspectives.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.