Second-order Stein: SURE for SURE and other applications in high-dimensional inference
Second-order Stein: SURE for SURE and other applications in high-dimensional inference
复制标题
DOI：
10.1214/20-aos2005
复制
发表时间：
2018-11
期刊：
影响因子：
--
通讯作者：
P. Bellec;Cun-Hui Zhang
中科院分区：
文献类型：
--
作者：
P. Bellec;Cun-Hui Zhang
斯坦因公式指出，对于所有具有可积梯度的函数 $f$，$z^\top f(z) - {\rm{div}} f(z)$ 形式的随机变量均值为零。这里，${\rm{div}} f$ 是函数$f$ 的散度，$z$ 是标准法向量。提出了二阶斯坦因公式来表征此类随机变量的方差。在高斯序列模型中，Stein 公式的一个显着结果是几乎任何给定估计量 $\hat\mu$ 对未知均值向量的均方风险的 Stein 无偏风险估计 (SURE)。二阶 Stein 公式的第一个应用是 SURE 本身风险的无偏风险估计（SURE for SURE）：简单的无偏估计提供有关 SURE 与 $\hat\mu$ 平方估计误差之间的平方距离的信息。 SURE for SURE 具有简单的形式，并且可以针对可微分 $\hat\mu$ 进行显式计算，例如套索和弹性网络。二阶斯坦因公式的其他应用在高维回归中提供。这包括套索选择的模型大小方差的新界限，以及对几乎可微的初始估计器进行去偏的通用半参数方案，以便估计未知回归系数向量的低维投影。
Stein's formula states that a random variable of the form $z^\top f(z) - {\rm{div}} f(z)$ is mean-zero for all functions $f$ with integrable gradient. Here, ${\rm{div}} f$ is the divergence of the function $f$ and $z$ is a standard normal vector. A Second Order Stein formula is proposed to characterize the variance of such random variables. In the Gaussian sequence model, a remarkable consequence of Stein's formula is Stein's Unbiased Risk Estimate (SURE) of the mean square risk of almost any given estimator $\hat\mu$ for the unknown mean vector. A first application of the Second Order Stein formula is an Unbiased Risk Estimate of the risk of SURE itself (SURE for SURE): a simple unbiased estimate provides information about the squared distance between SURE and the squared estimation error of $\hat\mu$. SURE for SURE has a simple form and can be computed explicitly for differentiable $\hat\mu$, for example the Lasso and the Elastic Net. Other applications of the Second Order Stein formula are provided in high-dimensional regression. This includes novel bounds on the variance of the size of the model selected by the Lasso, and a general semi-parametric scheme to de-bias an almost differentiable initial estimator in order to estimate a low-dimensional projection of the unknown regression coefficient vector.