Collaborative Research: Leverage Subsampling for Regression and Dimension Reduction

由于信息技术的快速发展，在科学、工程、社会科学、商业和政府的所有领域都产生了大量的数据集。通常通过统计模型拟合从这些数据中提取有用的信息，例如，回归模型。这些模型对于描述预测变量和响应变量之间的关系非常有用。给定一组n个数据元素和p个预测器，p和/或n在许多现代大规模数据集应用中可能很大。 在这些情况下，传统算法往往面临着严峻的计算挑战。数据矩阵的行和/或列的子采样传统上被用作减少大数据集的大小的启发式方法，从而使计算能够更快地运行。然而，最近，一种创新的抽样方法，使用的数据矩阵的经验统计杠杆分数作为一个非均匀的重要性抽样分布已经提出。 这已被应用于普通最小二乘（OLS）问题和其他相关问题，这种基于非均匀采样的方法给出了一个非常好的近似OLS的基础上充分的数据（当p是小的，n是大的）比传统的方法更快，无论是在最坏情况下的理论和高质量的数值实现。然而，到目前为止，这些算法的统计特性尚未被探索。 了解这些性质是有意义的，既有基本的原因，也有非常实际的原因;研究人员的工作解决了这些问题。 研究人员既考虑统计理论，也考虑在大量真实世界数据上用高质量的数值实现对该理论进行评估。本研究计划包括两个相关的研究方向，这两个方向都围绕着综合处理统计和计算问题的共同目标。 第一个研究重点是利用线性回归中的统计杠杆分数来研究子抽样估计的统计特性。第二个研究方向是将理论和方法推广到非线性回归和降维模型。这些理论和方法的提出为统计方法学的发展提供了新的思路。该研究为现有算法提供了新的见解，产生了用于分析大规模数据的创新方法，激发了跨学科研究中新的定量研究路线，并提供了独特的教育体验。

As a result of rapid advances in information technology, massive datasets are being generated in all fields of science, engineering, social science, business, and government. Useful information is often extracted from these data through statistical model fitting, e.g., through regression models. These models are useful for describing relationships between predictor variables and a response variable. Given a set of n data elements and p predictors, p and/or n can be large in much modern massive data set applications. In these cases, conventional algorithms often face severe computational challenges. Subsampling of rows and/or columns of a data matrix have traditionally been employed as a heuristic to reduce the size of large data sets, thus enabling computations to run more quickly. Recently, however, an innovative sampling methodology that uses the empirical statistical leverage scores of the data matrix as a nonuniform importance sampling distribution has been proposed. This has been applied to the ordinary least squares (OLS) problem and other related problems, and this leverage-based nonuniform sampling procedure gives a very good approximation to the OLS based on full data (when p is small and n is large) more rapidly than traditional methods, both in worst-case theory and in high-quality numerical implementations. As of yet, however, the statistical properties of these algorithms are unexplored. Understanding these properties is of interest for both fundamental and very practical reasons; and the investigators' work addresses these problems. The investigators consider both statistical theory as well as the evaluation of that theory with high-quality numerical implementations on large real-world data.This research proposal consists of two related research thrusts, both of which center around the common goal of an integrated treatment of statistical and computational issues. The first research thrust focuses on studying the statistical properties of the subsampling estimation using the statistical leverage scores in linear regression. The second research thrust generalizes the theory and methods to nonlinear regression and dimension reduction models. The proposed theory and methods serve as an inspiration for new ideas to push statistical methodology development forward. The research provides new insight into the existing algorithms, produces innovative methodologies for analyzing large-scale data, inspires new lines of quantitative investigations in interdisciplinary research and offers a unique educational experience.