Hypothesis Testing in High-Dimensional Regression Under the Gaussian Random Design Model: Asymptotic Theory
Hypothesis Testing in High-Dimensional Regression Under the Gaussian Random Design Model: Asymptotic Theory
复制标题
DOI：
10.1109/tit.2014.2343629
复制
发表时间：
2014-10-01
影响因子：
2.5
通讯作者：
Montanari, Andrea
中科院分区：
文献类型：
--
作者：
Javanmard, Adel;Montanari, Andrea
我们考虑高维区域中的线性回归，其中观测数n小于参数数p。在这种设置中，一种非常成功的方法使用l（1）-惩罚最小二乘法（也称为Lasso）来搜索s（0）< n的参数子集，这些参数最好地解释数据，同时将其他参数设置为零。大量的工作已被用于描述这种方法中的估计和模型选择问题。在本文中，我们考虑的是基本的，但远未理解，统计意义的问题。更确切地说，我们解决了计算单个回归系数的p值的问题。一方面，我们在给定的显著性水平下，给出了极大极小检验功效的一般上界。我们表明，严格的保证早期的方法不允许实现这一界限，除非在特殊情况下。另一方面，我们证明了这个上界是（几乎）通过一个实际的过程中的情况下，随机设计矩阵的独立项目是可以实现的。我们的方法是基于一个去偏的Lasso估计。分析建立在严格表征的渐近分布的Lasso估计及其去偏版本。我们的结果适用于最佳样本量，即，当n至少为s（0）log（p/s（0））的量级时。我们将我们的方法推广到具有独立同分布高斯行x（i）的随机设计矩阵，其类似于N（0，Sigma）。在这种情况下，我们证明了一个类似的分布特征（称为标准分布极限）适用于n远大于s（0）（log p）（2）。我们的分析假设Sigma是已知的。为了科普未知的Sigma，我们提出了一个插件估计稀疏协方差Sigma，并通过数值模拟验证该方法。最后，我们证明了对于最优样本容量，n至少为s（0）log（p/s（0））阶，一般高斯设计的标准分布极限可以从统计物理学中的副本统计学推导出来。这个推导表明了一个比我们证明的结果更强的猜想，以及一大类高斯设计的统计功效的近优性。
We consider linear regression in the high-dimensional regime where the number of observations n is smaller than the number of parameters p. A very successful approach in this setting uses l(1)-penalized least squares (also known as the Lasso) to search for a subset of s(0) < n parameters that best explain the data, while setting the other parameters to zero. Considerable amount of work has been devoted to characterizing the estimation and model selection problems within this approach. In this paper, we consider instead the fundamental, but far less understood, question of statistical significance. More precisely, we address the problem of computing p-values for single regression coefficients. On one hand, we develop a general upper bound on the minimax power of tests with a given significance level. We show that rigorous guarantees for earlier methods do not allow to achieve this bound, except in special cases. On the other, we prove that this upper bound is (nearly) achievable through a practical procedure in the case of random design matrices with independent entries. Our approach is based on a debiasing of the Lasso estimator. The analysis builds on a rigorous characterization of the asymptotic distribution of the Lasso estimator and its debiased version. Our result holds for optimal sample size, i.e., when n is at least on the order of s(0) log(p/s(0)). We generalize our approach to random design matrices with independent identically distributed Gaussian rows x(i) similar to N(0, Sigma). In this case, we prove that a similar distributional characterization (termed standard distributional limit) holds for n much larger than s(0)(log p)(2). Our analysis assumes Sigma is known. To cope with unknown Sigma, we suggest a plug-in estimator for sparse covariances Sigma and validate the method through numerical simulations. Finally, we show that for optimal sample size, n being at least of order s(0) log(p/s(0)), the standard distributional limit for general Gaussian designs can be derived from the replica heuristics in statistical physics. This derivation suggests a stronger conjecture than the result we prove, and near-optimality of the statistical power for a large class of Gaussian designs.