Collaborative research: Statistical and computational efficiency for massive data sets via approximation-regularization

该项目将计算机科学中的近似方法与现代统计理论相结合，以改善大型数据集的分析。现代统计分析需要在大型数据集上计算可行的方法，同时保持统计效率。通常，这两个问题被认为是矛盾的：近似方法，使计算被认为是降低统计性能相对于精确的方法。从统计学的角度来看，精确解是不可取的，正则化解是首选。正则化可以被认为是对数据的保真度和遵守有关数据生成过程的先验知识（如平滑性或稀疏性）之间的权衡。由此产生的估计量往往更有用，更可解释，更适合作为其他方法的输入。相反，在计算机科学应用中，目前大部分近似方法的工作都存在，输入通常被认为是精确观察的。流行的哲学是，虽然确切的问题是，遗憾的是，无法解决，任何近似的解决方案应该尽可能接近的确切之一。我们有一个重要的认识：近似方法本身自然会导致正则化，这表明了一种有趣的可能性，即一些计算近似可以同时分析大量数据，同时提高统计性能。我们的研究开发了利用这种现象的新方法，我们称之为“近似正则化”。第一种方法使用矩阵预处理器来稳定最小二乘准则。如果正确校准，这种方法提供了计算和存储的优势，正则化最小二乘法，同时提供了一个统计上的上级解决方案。第二个创新解决了主成分分析（PCA）的大型数据集的回归，其中PCA是计算上不可行的，已知是统计上不一致的。通过采用随机近似，我们可以解决这两个问题，同时改善预测。最后，我们介绍了新的方法，无监督降维，从而近似算法，利用稀疏性，统计方法，诱导它，使频谱技术的使用非常大的矩阵。在每一种情况下，近似正则化产生相对于现有方法的计算和统计增益。这项研究认识到，近似是正则化，从而可以提高统计精度，同时使计算。它将导致为大型数据集制定新的统计方法，这些方法在计算和统计上都优于现有方法，同时也引起对统计中这一重要领域的关注。此外，这些方法将使天文学、遗传学、文本和图像处理、气候科学和预测等其他领域的科学家能够随时利用现有数据。

This project integrates approximation methodology from computer science with modern statistical theory to improve analysis of large data sets. Modern statistical analysis requires methods that are computationally feasible on large datasets while at the same time preserving statistical efficiency. Frequently, these two concerns are seen as contradictory: approximation methods that enable computation are assumed to degrade statistical performance relative to exact methods. The statistical perspective is that the exact solution is undesirable, and a regularized solution is preferred. Regularization can be thought of as formalizing a trade-off between fidelity to the data and adherence to prior knowledge about the data-generating process such as smoothness or sparsity. The resulting estimator tends to be more useful, interpretable, and suitable as an input to other methods. Conversely, in computer science applications, where much of the current work on approximation methods resides, the inputs are generally considered to be observed exactly. The prevailing philosophy is that while the exact problem is, regrettably, unsolvable, any approximate solution should be as close as possible to the exact one. We make a crucial realization: that the approximation methods themselves naturally lead to regularization, suggesting the intriguing possibility that some computational approximations can simultaneously enable the analysis of massive data while enhancing statistical performance. Our research develops new methods that leverage this phenomenon, which we have dubbed 'approximation-regularization.' The first method uses a matrix pre-conditioner to stabilize the least-squares criterion. If properly calibrated, this approach provides computational and storage advantages over regularized least squares while providing a statistically superior solution. A second innovation addresses principal components analysis (PCA) for regression on large data sets where PCA is both computationally infeasible and known to be statistically inconsistent. By employing randomized approximations, we can address both of these issues, while improving predictions at the same time. Lastly, we introduce new methods for unsupervised dimension reduction, whereby approximation algorithms that leverage sparsity, and statistical methods that induce it, enable the use of spectral techniques on very large matrices. In each of these cases, approximation-regularization yields both computational and statistical gains relative to existing methodologies. This research recognizes that approximation is regularization and can thereby increase statistical accuracy while enabling computation. It will result in new statistical methods for large datasets, which are computationally and statistically preferable to existing approaches, while also bringing attention to this important area in statistics. Additionally, these methods will permit scientists in other fields, such as astronomy, genetics, text and image processing, climate science, and forecasting, to make ready use of available data.