Collaborative Research: Leverage Subsampling for Regression and Dimension Reduction

由于信息技术的快速发展，在科学、工程、社会科学、商业和政府的各个领域都产生了海量的数据集。通常通过统计模型拟合，例如通过回归模型，从这些数据中提取有用的信息。这些模型对于描述预测变量和响应变量之间的关系很有用。在给定n个数据元素和p个预测器的集合的情况下，p和/或n在许多现代海量数据集应用中可能很大。在这些情况下，传统算法通常面临严峻的计算挑战。传统上，数据矩阵的行和/或列的二次采样被用作减少大型数据集的大小的启发式方法，从而使计算能够更快地运行。然而，最近提出了一种创新的抽样方法，该方法使用数据矩阵的经验统计杠杆分数作为非均匀重要性抽样分布。这已经被应用于普通最小二乘(OLS)问题和其他相关问题，并且这种基于杠杆的非均匀抽样过程在最坏情况理论和高质量的数值实现中都比传统方法更快地逼近基于全数据(当p小而n大)的普通最小二乘问题。然而，到目前为止，这些算法的统计特性还没有得到探索。理解这些性质是有意义的，既有基本的，也有非常实际的原因；调查人员的工作解决了这些问题。研究人员既考虑了统计理论，也考虑了对该理论的评估，并对大量真实世界数据进行了高质量的数值实现。这项研究方案包括两个相关的研究推动力，两者都围绕着综合处理统计和计算问题的共同目标。第一个研究重点是利用线性回归中的统计杠杆分数来研究欠抽样估计的统计性质。第二个研究重点是将理论和方法推广到非线性回归和降维模型。所提出的理论和方法为推动统计方法论发展提供了新的思路。这项研究为现有算法提供了新的见解，为分析大规模数据提供了创新的方法，激发了跨学科研究中定量研究的新思路，并提供了独特的教育体验。

As a result of rapid advances in information technology, massive datasets are being generated in all fields of science, engineering, social science, business, and government. Useful information is often extracted from these data through statistical model fitting, e.g., through regression models. These models are useful for describing relationships between predictor variables and a response variable. Given a set of n data elements and p predictors, p and/or n can be large in much modern massive data set applications. In these cases, conventional algorithms often face severe computational challenges. Subsampling of rows and/or columns of a data matrix have traditionally been employed as a heuristic to reduce the size of large data sets, thus enabling computations to run more quickly. Recently, however, an innovative sampling methodology that uses the empirical statistical leverage scores of the data matrix as a nonuniform importance sampling distribution has been proposed. This has been applied to the ordinary least squares (OLS) problem and other related problems, and this leverage-based nonuniform sampling procedure gives a very good approximation to the OLS based on full data (when p is small and n is large) more rapidly than traditional methods, both in worst-case theory and in high-quality numerical implementations. As of yet, however, the statistical properties of these algorithms are unexplored. Understanding these properties is of interest for both fundamental and very practical reasons; and the investigators' work addresses these problems. The investigators consider both statistical theory as well as the evaluation of that theory with high-quality numerical implementations on large real-world data.This research proposal consists of two related research thrusts, both of which center around the common goal of an integrated treatment of statistical and computational issues. The first research thrust focuses on studying the statistical properties of the subsampling estimation using the statistical leverage scores in linear regression. The second research thrust generalizes the theory and methods to nonlinear regression and dimension reduction models. The proposed theory and methods serve as an inspiration for new ideas to push statistical methodology development forward. The research provides new insight into the existing algorithms, produces innovative methodologies for analyzing large-scale data, inspires new lines of quantitative investigations in interdisciplinary research and offers a unique educational experience.