Learning with Confidence: Bootstrapping Error Estimates for Stochastic Iterative Algorithms

在这个大规模高维数据分析的时代，随机迭代优化方法构成了一类流行的算法，它一次只查看一个数据点。这些算法越来越多地被赋予决策任务，从丢弃垃圾邮件到推荐开车上班的路线，再到自动驾驶汽车和用于诊断目的的医学图像的未来决策。这些应用会产生由测量误差和实验设置或环境的可变性引起的噪声数据。 尽管如此，目前几乎所有的理论结果都集中在基本参数的准确估计上。然而，重要的是要注意量化学习参数的可变性的置信区间，这使得人们能够有信心地做出决策。本研究将发展一个数学架构，以估计一大类随机迭代最佳化方法的不确定性。该项目还将帮助培训研究生和博士后学者对大规模学习问题的不确定性估计，从而使他们更好地为工业和学术界的职业生涯做好准备。研究人员将建立基本随机迭代学习算法的中心极限定理和一致的在线引导程序，将这些算法纳入软件包，并开发一个框架，用于评估广泛应用中点估计预测的置信度。这些目标将通过使用各种最新开发的工具来实现，包括随机矩阵乘积的浓度不等式、高维统计和深度学习优化方法的理论进展。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In this age of large-scale high-dimensional data analysis, stochastic iterative optimization methods constitute a popular class of algorithms, which look at one data point at a time. These algorithms are increasingly entrusted with decision-making tasks, from discarding emails as spam to recommending routes when one drives to work, to the future decisions made by self-driving cars and medical images produced for diagnostic purposes. These applications give rise to noisy data resulting from measurement errors and variability in the experimental setup or environment. Despite this, almost all current theoretical results focus on the accurate estimation of underlying parameters. However, it is crucial to pay attention to confidence intervals that quantify the variability of the learned parameters, which allows one to make decisions with confidence. This research will develop a mathematical framework for the estimation of uncertainty of a broad class of stochastic iterative optimization methods. This project will also help train graduate students and postdoctoral scholars in uncertainty estimation for large-scale learning problems, thus making them better prepared for careers in both industry and academia. The investigators will establish central limit theorems and consistent online bootstrap procedures for fundamental stochastic iterative learning algorithms, incorporate these algorithms in software packages, and develop a framework for assessing confidence in point estimate predictions in a broad range of applications. These goals will be achieved using a variety of recently developed tools, including concentration inequalities for products of random matrices, high dimensional statistics, and theoretical advances in deep learning optimization methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.