Iterative Algorithms for Statistics: From Convergence Rates to Statistical Accuracy

现代数据科学时代正在给科学、工程和工业带来革命性的变化，在这个时代，人们可以获得越来越大和丰富的数据形式。这些应用多种多样，意义广泛，包括天文学中的数据驱动发现，药物设计的统计机器学习方法，以及机器人学和自动驾驶方面的决策等。这笔赠款支持研究从如此庞大的数据集中学习的技术和模型，从而产生计算效率高的算法，这些算法可以扩展到实践中遇到的大型问题实例。国际和平研究所计划通过研究生参与研究，将研究成果纳入加州大学伯克利分校的课程和公开提供的网络课程材料，以及暑期学校和讲习班的小型课程，将研究与教育结合起来。这个项目还将为女性研究生和博士后提供指导和支持。统计学中的许多估计是通过应用于依赖数据的目标函数的迭代算法来定义的(例如，用于缺失数据和潜变量模型的EM算法；用于M估计的基于梯度的方法和牛顿方法；用于非参数回归的增强算法)。这个项目给出了几个研究方向，围绕着利用这些算法的动态来回答统计问题，并应用于统计参数估计；混合模型中组件数量的选择；以及非参数回归中的最优偏差-方差权衡。更详细地说，本项目的目标包括：(I)对非规则混合模型和相关奇异问题的EM算法进行一般分析，其中通常观察到非常慢的(亚几何)收敛；(Ii)发展基于EM收敛速度的模型选择的原则性方法，并证明其性能的理论保证；发展一个将算法的收敛速度与其(In)稳定性的界相结合的一般理论框架，以建立统计估计误差的界；以及(Iii)为各种类型的增强更新(包括内核增强以及梯度增强回归树)提供完整的增强路径分析，并分析“过度匹配”机制，阐明发生或不发生过度匹配的条件。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Science, engineering, and industry are all being revolutionized by the modern era of data science, in which increasingly large and rich forms of data are now available. The applications are diverse and broadly significant, including data-driven discovery in astronomy, statistical machine learning approaches to drug design, and decision-making in robotics and automated driving, among many others. This grant supports research on techniques and models for learning from such massive datasets, leading to computationally efficient algorithms that can be scaled to the large problem instances encountered in practice. The PI plans to integrate research and education through the involvement of graduate students in the research, the inclusion of the research results in courses at UC Berkeley and in publicly available web-based course materials, as well as in mini courses at summer schools and workshops. This project will also provide mentoring and support for graduate students and postdocs who are female or belong to URM communities.Many estimates in statistics are defined via an iterative algorithm applied to a data-dependent objective function (e.g., the EM algorithm for missing data and latent variable models; gradient-based methods and Newton's method for M-estimation; boosting algorithms used in non-parametric regression). This projectl gives several research thrusts that are centered around exploiting the dynamics of these algorithms in order to answer statistical questions, with applications to statistical parameter estimation; selection of the number of components in a mixture model; and optimal bias-variance trade-offs in non-parametric regression. In more detail, the aims of this project include (i) providing a general analysis of the EM algorithm for non-regular mixture models and related singular problems, in which very slow (sub-geometric) convergence is typically observed; (ii) developing a principled method for model selection based on the convergence rate of EM, and to prove theoretical guarantees on its performance; developing a general theoretical framework for combining the convergence rate of an algorithm with bounds on its (in)stability so as to establish bounds on the statistical estimation error; and (iii) providing a complete analysis of the full boosting path for various types of boosting updates, including kernel boosting, as well as gradient-boosted regression trees, and to analyze the "overfitting" regime, elucidating conditions under which overfitting does or does not occur.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.