Collaborative Research: Leverage Subsampling for Regression and Dimension Reduction

由于信息技术的快速发展，在科学、工程、社会科学、商业和政府的各个领域都产生了大量的数据集。通常通过统计模型拟合，例如通过回归模型，从这些数据中提取有用的信息。这些模型用于描述预测变量和响应变量之间的关系。给定一组n个数据元素和p个预测器，p和/或n在许多现代大规模数据集应用程序中可能很大。在这些情况下，传统算法往往面临严峻的计算挑战。对数据矩阵的行和/或列进行子采样传统上被用作一种启发式方法，以减少大型数据集的大小，从而使计算运行得更快。然而，最近提出了一种创新的抽样方法，该方法使用数据矩阵的经验统计杠杆分数作为非均匀重要性抽样分布。这已经应用于普通最小二乘（OLS）问题和其他相关问题，并且这种基于杠杆的非均匀抽样过程在最坏情况理论和高质量数值实现中，比传统方法更快地提供了基于完整数据（当p很小而n很大时）的非常好的OLS近似。然而，到目前为止，这些算法的统计特性尚未被探索。理解这些属性既有基本的原因，也有非常实际的原因；调查人员的工作解决了这些问题。研究人员考虑统计理论以及该理论的评估与高质量的数值实现在大型现实世界的数据。本研究计划由两个相关的研究重点组成，它们都围绕统计和计算问题的综合处理这一共同目标。第一个研究重点是利用线性回归中的统计杠杆分数研究子抽样估计的统计性质。第二部分将理论和方法推广到非线性回归和降维模型。所提出的理论和方法为推动统计方法学的发展提供了新的思路。该研究提供了对现有算法的新见解，产生了分析大规模数据的创新方法，激发了跨学科研究中定量调查的新思路，并提供了独特的教育体验。

As a result of rapid advances in information technology, massive datasets are being generated in all fields of science, engineering, social science, business, and government. Useful information is often extracted from these data through statistical model fitting, e.g., through regression models. These models are useful for describing relationships between predictor variables and a response variable. Given a set of n data elements and p predictors, p and/or n can be large in much modern massive data set applications. In these cases, conventional algorithms often face severe computational challenges. Subsampling of rows and/or columns of a data matrix have traditionally been employed as a heuristic to reduce the size of large data sets, thus enabling computations to run more quickly. Recently, however, an innovative sampling methodology that uses the empirical statistical leverage scores of the data matrix as a nonuniform importance sampling distribution has been proposed. This has been applied to the ordinary least squares (OLS) problem and other related problems, and this leverage-based nonuniform sampling procedure gives a very good approximation to the OLS based on full data (when p is small and n is large) more rapidly than traditional methods, both in worst-case theory and in high-quality numerical implementations. As of yet, however, the statistical properties of these algorithms are unexplored. Understanding these properties is of interest for both fundamental and very practical reasons; and the investigators' work addresses these problems. The investigators consider both statistical theory as well as the evaluation of that theory with high-quality numerical implementations on large real-world data. This research proposal consists of two related research thrusts, both of which center around the common goal of an integrated treatment of statistical and computational issues. The first research thrust focuses on studying the statistical properties of the subsampling estimation using the statistical leverage scores in linear regression. The second research thrust generalizes the theory and methods to nonlinear regression and dimension reduction models. The proposed theory and methods serve as an inspiration for new ideas to push statistical methodology development forward. The research provides new insight into the existing algorithms, produces innovative methodologies for analyzing large-scale data, inspires new lines of quantitative investigations in interdisciplinary research and offers a unique educational experience.