Theory and Methods for Modern Predictive Inference

在统计学和机器学习中，一个核心的推理任务是根据过去的观察结果预测未来数据的值。预测算法在日常生活中被广泛使用，如垃圾邮件检测、健康风险评估、天气预报、经济等，使其成为统计推断任务中最基本和最重要的类别之一。在过去的十年中，深度神经网络和强大计算机的出现在设计和实现预测算法方面取得了重大进展，导致方法和应用的爆炸性发展。这些新的应用程序要求新的统计原则的方法与数学的理由，充分和正确地利用这些新工具的力量。 然而，算法和数据集中不可见的复杂程度对依赖于简单问题结构的经典统计和学习理论框架构成了根本挑战。 受现代数据科学时代算法和计算进步的推动，在本研究项目中，我们计划承担几个新的方法挑战，并填补统计预测推理的理论空白，包括对大量预测模型的同时准确性评估，以及使用校准预测进行有效的统计推理。该项目还为研究生提供研究培训机会。本研究项目由两部分组成。第一部分将提供交叉验证的新见解，交叉验证是模型和调优参数选择中最广泛使用的方法之一。该理论的发展将有助于理解交叉验证风险的联合随机性，不仅解释了广泛观察到的交叉验证过拟合趋势，还指出了纠正它的潜在解决方案，这项研究工作将进一步丰富和连接多个活跃的研究领域，包括交叉验证，高维高斯比较，模型置信集和在线学习。第二部分的目的是填补一个重要的空白，在共形预测文献开发的共形为基础的假设检验方法超越互换性。我们计划探索共形预测和经典主题之间的联系，如曼-惠特尼秩和统计，两样本U-统计，半参数推断。预期的结果将导致新的应用共形推理和创造一个新的阵列的研究问题，在这些相关的研究topics.This奖项反映了NSF的法定使命，并已被认为是值得的支持，通过评估使用基金会的智力价值和更广泛的影响审查标准。

In statistics and machine learning, a central inferential task is to predict the values of future data given past observations. Prediction algorithms are widely used on a daily basis, such as spam detection, health risk evaluation, weather forecasting, economy, etc., making it one of the most fundamental and important classes of statistical inference tasks. In the past decade, the emergence of deep neural networks and powerful computers have made major progress in designing and implementing prediction algorithms, resulting in an explosive development of methods and applications. These new applications call for novel statistically principled methods with mathematical justifications to fully and correctly exploit the power of such new tools. However, the unseen level of complexity in both the algorithms and datasets poses fundamental challenges to classical statistical and learning-theoretical frameworks that rely on simple problem structures. Motivated by the algorithmic and computational advances in the modern data science era, in this research project, we plan to take on several new methodological challenges and fill theoretical gaps in statistical predictive inference, including simultaneous accuracy evaluation for a large collection of prediction models, and valid statistical inference using calibrated prediction. The project also provides research training opportunities for graduate students. The research project consists of two parts. The first part will provide new insights into cross-validation, one of the most widely used methods for model and tuning parameter selection. The theoretical development will contribute to the understanding of the joint randomness of cross-validated risks, not only explaining the widely observed overfitting tendency of cross-validation but also pointing out potential solutions to correct it. This research work will further enrich and connect multiple areas of active research, including cross-validation, high-dimensional Gaussian comparison, model confidence set, and online learning. The second part aims at filling an important gap in the conformal prediction literature by developing a conformal-based hypothesis testing method beyond ex-changeability. We plan to explore connections between conformal prediction and classical topics such as Mann-Whitney rank-sum statistic, two-sample U-statistics, and semiparametric inference. The expected results will lead to new applications of conformal inference and create a new array of research problems across these related research topics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.