Collaborative Research: Consistent Risk Estimation under High-Dimensional Asymptotics

从大数据集中学习一直是科学、医学和技术领域现代创新和发现的基石。对未知事件的快速预测是统计学学习中的一个典型目标。达到这一目的的经典方法是留一交叉验证，这是一个耗时的例程，需要省去一个基准，将模型拟合到其余的基准上，然后反复在剩下的基准上进行测试。最近出现的大量数据加剧了这些方法在计算上的不可行性。此外，在最近的许多情况下，每个观测的特征数量可能非常大，这给预测误差的快速估计增加了另一个具有挑战性的方面。为了克服这些问题，本项目将开发一套新的可扩展和一致的风险估计器。风险评估的重要性促使本项目采用了不同的方案，如交叉验证、Stein的无偏风险估计(SURE)、广义交叉验证、Akaike信息准则(AIC)和Bootstrap。高维数据集的出现对大多数经典的风险估计方法提出了挑战。例如，在涉及基于以前未见的特征的预测的应用中，样本内和样本外预测误差之间的巨大差异，使得在预测器的数量小于或与观测值的数量相同的高维区域中，很难依赖流行的估计器，例如SURE或AIC。另一方面，这些机制中的数据的信息值(与低维环境中的数据的信息值相反)使人对其他技术的可靠性产生怀疑，例如5次交叉验证。该项目提供了一个新的理论框架来寻找可伸缩性和可靠性之间的中间地带，具体地说，在高维环境下获得理论上一致和计算上有效的风险估计方案。由于风险评估是包括但不限于机器学习、信号处理、医学成像、神经科学以及社会科学和环境科学等领域的核心，该项目的任何成功都将带来可靠和即时的科学发现和更好的学习系统。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Learning from large datasets has been the cornerstone of modern innovations and discoveries in science, medicine, and technology. Fast prediction of unseen events is a canonical goal in statistical learning. A classic approach to this end is leave-one-out cross-validation, a time-consuming routine of leaving a datum out, fitting the model on the rest, and testing it on the left out datum, repeatedly. The recent emergence of massive data has exacerbated the computational infeasibility of such approaches. Moreover, in many recent instances, the number of features per observation can be extremely large, adding another challenging facet to the fast estimation of prediction error. To overcome these problems a new set of scalable and consistent risk estimators will be developed in this project. The importance of risk estimation has motivated this project of different schemes, such as cross-validation, Stein's unbiased risk estimation (SURE), Generalized cross-validation, Akaike Information Criterion (AIC), and Bootstrap. The emergence of high-dimensional datasets has challenged most classical approaches to risk estimation. For instance, the large discrepancy between in-sample and out-of-sample prediction error, in applications involving predictions based on previously unseen features, makes it hard to rely on popular estimators, such as SURE or AIC, in high-dimensional regimes where the number of predictors is smaller than or at the same order as the number of observations. On the other hand, the information value of a datum in these regimes (as opposed to the information value of a datum in low-dimensional settings) casts doubt on the reliability of other techniques, such as 5-fold cross-validation. The project offers a novel theoretical framework to find the middle ground between scalability and reliability, and specifically, to obtain theoretically consistent and computationally efficient risk-estimation schemes under high-dimensional settings. Since risk estimation is at the core of areas including but not limited to machine learning, signal processing, medical imaging, neuroscience, and social and environmental sciences, any success in this project will lead to reliable and immediate scientific discoveries and better learning systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.