Interpolation Methods in Statistics and Machine Learning

统计和机器学习中的插值方法

• 批准号: 1953181
• 负责人: Alexander Rakhlin
• 金额: $ 20万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-01 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1953181&HistoricalAwards=false
• 关键词: Interpolation; Methods; Statistics; Machine; Learning

One of the key tenets taught in courses on Statistics and Machine Learning is that data interpolation (or, data memorization) inevitably leads to overfitting and poor prediction performance. Yet, most of the modern large-scale models, including over-parametrized neural networks, are routinely optimized to achieve zero error on training data. The research objective of this project is to challenge the common wisdom and develop theoretical and algorithmic foundations for methods that interpolate the training data. The project will focus on the statistical and computational aspects of interpolation methods. Consistency and finite-sample bounds will be derived for regression and classification methods in the interpolation regime, and information-theoretic limits of interpolating rules will be developed. The project will also focus on the computational aspects of interpolation. The PI aims to shed light on the relative advantages and disadvantages of over-parametrized models that have capacity to perfectly fit the data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

统计学和机器学习课程中教授的关键原则之一是数据插值（或数据记忆）不可避免地导致过拟合和预测性能差。然而，大多数现代大规模模型，包括过度参数化的神经网络，通常都是为了实现训练数据的零误差而进行优化的。该项目的研究目标是挑战常识，并为插值训练数据的方法建立理论和算法基础。 该项目将侧重于插值方法的统计和计算方面。一致性和有限样本的界限将推导出回归和分类方法的插值制度，和信息理论的限制插值规则将开发。该项目还将侧重于插值的计算方面。PI旨在阐明过度参数化模型的相对优势和劣势，这些模型具有完美拟合数据的能力。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

On the Minimal Error of Empirical Risk Minimization

论经验风险最小化的最小误差

• 发表时间: 2021
• 期刊: Conference on Learning Theory
• 作者: Gil Kur, Alexander Rakhlin

Intrinsic Dimension Estimation Using Wasserstein Distance

• 发表时间: 2021-06
• 期刊: J. Mach. Learn. Res.
• 作者: A. Block;Zeyu Jia;Yury Polyanskiy;A. Rakhlin

Deep learning: a statistical viewpoint

• DOI: 10.1017/s0962492921000027
• 发表时间: 2021-05-01
• 期刊: ACTA NUMERICA
• 影响因子: 14.2
• 作者: Bartlett, Peter L.;Montanari, Andrea;Rakhlin, Alexander
• 通讯作者: Rakhlin, Alexander

On Suboptimality of Least Squares with Application to Estimation of Convex Bodies

最小二乘法的次优性及其在凸体估计中的应用

• 发表时间: 2020
• 期刊: PMLR
• 作者: Kur, Gil;Rakhlin, Alexander;Guntuboyina, Adityanand
• 通讯作者: Guntuboyina, Adityanand

An Efficient Minimax Optimal Estimator For Multivariate Convex Regression

多元凸回归的高效极小极大最优估计器

• 发表时间: 2022
• 期刊: Conference on Learning Theory
• 作者: Kur, G;Putterman, E
• 通讯作者: Putterman, E