A projector-based approach to quantifying total and excess uncertainties for sketched linear regression
A projector-based approach to quantifying total and excess uncertainties for sketched linear regression
复制标题
DOI：
10.1093/imaiai/iaab016
复制
发表时间：
2021-08-11
影响因子：
1.6
通讯作者：
Ipsen, Ilse C. F.
中科院分区：
文献类型：
--
作者：
Chi, Jocelyn T.;Ipsen, Ilse C. F.
线性回归是数据分析的经典方法。近年来，草图（一种使用随机抽样、随机投影或两者的降维方法）作为观察数量大大超过变量数量时的有效计算近似而受到欢迎。在本文中，我们解决了以下问题：素描如何影响的解决方案和关键量的统计特性？为了回答这个问题，我们提出了一个基于投影仪的方法来勾画线性回归，这是准确的，需要最少的假设上的草图矩阵。因此，下游分析对所有草图方案都准确且普遍适用。此外，基于投影仪的方法能够从经典线性回归中推导出关键量，这些回归考虑了模型和算法引起的不确定性。我们展示了一个基于投影仪的方法在量化和实现对勾画线性回归的过度不确定性和偏差方差分解的洞察方面的有用性。最后，我们展示了如何从我们的基于投影仪的分析的见解可以用来产生实际的草图诊断，以帮助明智的草图方案的设计。
Linear regression is a classic method of data analysis. In recent years, sketching-a method of dimension reduction using random sampling, random projections or both-has gained popularity as an effective computational approximation when the number of observations greatly exceeds the number of variables. In this paper, we address the following question: how does sketching affect the statistical properties of the solution and key quantities derived from it? To answer this question, we present a projector-based approach to sketched linear regression that is exact and that requires minimal assumptions on the sketching matrix. Therefore, downstream analyses hold exactly and generally for all sketching schemes. Additionally, a projector-based approach enables derivation of key quantities from classic linear regression that account for the combined model- and algorithm-induced uncertainties. We demonstrate the usefulness of a projector-based approach in quantifying and enabling insight on excess uncertainties and bias-variance decompositions for sketched linear regression. Finally, we demonstrate how the insights from our projector-based analyses can be used to produce practical sketching diagnostics to aid the design of judicious sketching schemes.