Collaborative Research: Consistent Risk Estimation under High-Dimensional Asymptotics

从大型数据集中学习一直是现代科学、医学和技术创新和发现的基石。对未知事件的快速预测是统计学习的一个典型目标。达到这一目的的一个经典方法是“留一”交叉验证，这是一个耗时的例程，需要留下一个数据，在剩下的数据上拟合模型，然后在留下的数据上反复测试它。最近大量数据的出现加剧了这种方法在计算上的不可行性。此外，在最近的许多情况下，每次观测的特征数量可能非常大，这给预测误差的快速估计增加了另一个挑战。为了克服这些问题，本项目将开发一套新的可伸缩和一致的风险评估器。风险估计的重要性促使本项目采用了交叉验证、Stein’s无偏风险估计（SURE）、广义交叉验证、赤池信息准则（AIC）和Bootstrap等方案。高维数据集的出现对大多数传统的风险估计方法提出了挑战。例如，在涉及基于先前未见特征的预测的应用程序中，样本内和样本外预测误差之间的巨大差异使得难以依赖流行的估计器，例如SURE或AIC，在高维体系中，预测器的数量小于或与观测数相同。另一方面，在这些制度中基准的信息价值（与低维设置中基准的信息价值相反）使人们对其他技术的可靠性产生怀疑，例如5倍交叉验证。该项目提供了一个新的理论框架，以找到可扩展性和可靠性之间的中间地带，特别是在高维设置下获得理论上一致和计算效率高的风险估计方案。由于风险评估是包括但不限于机器学习、信号处理、医学成像、神经科学、社会和环境科学等领域的核心，因此该项目的任何成功都将导致可靠和直接的科学发现和更好的学习系统。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Learning from large datasets has been the cornerstone of modern innovations and discoveries in science, medicine, and technology. Fast prediction of unseen events is a canonical goal in statistical learning. A classic approach to this end is leave-one-out cross-validation, a time-consuming routine of leaving a datum out, fitting the model on the rest, and testing it on the left out datum, repeatedly. The recent emergence of massive data has exacerbated the computational infeasibility of such approaches. Moreover, in many recent instances, the number of features per observation can be extremely large, adding another challenging facet to the fast estimation of prediction error. To overcome these problems a new set of scalable and consistent risk estimators will be developed in this project. The importance of risk estimation has motivated this project of different schemes, such as cross-validation, Stein's unbiased risk estimation (SURE), Generalized cross-validation, Akaike Information Criterion (AIC), and Bootstrap. The emergence of high-dimensional datasets has challenged most classical approaches to risk estimation. For instance, the large discrepancy between in-sample and out-of-sample prediction error, in applications involving predictions based on previously unseen features, makes it hard to rely on popular estimators, such as SURE or AIC, in high-dimensional regimes where the number of predictors is smaller than or at the same order as the number of observations. On the other hand, the information value of a datum in these regimes (as opposed to the information value of a datum in low-dimensional settings) casts doubt on the reliability of other techniques, such as 5-fold cross-validation. The project offers a novel theoretical framework to find the middle ground between scalability and reliability, and specifically, to obtain theoretically consistent and computationally efficient risk-estimation schemes under high-dimensional settings. Since risk estimation is at the core of areas including but not limited to machine learning, signal processing, medical imaging, neuroscience, and social and environmental sciences, any success in this project will lead to reliable and immediate scientific discoveries and better learning systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.