Consistent Parameter Estimation for LASSO and Approximate Message Passing
Consistent Parameter Estimation for LASSO and Approximate Message Passing
复制标题
LASSO 的一致参数估计和近似消息传递
DOI：
10.1214/17-aos1544
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Richard Baraniuk
中科院分区：
文献类型：
--
作者：
A. Mousavi;A. Maleki;Richard Baraniuk
我们考虑了从$n$随机和噪声线性观测值$y=X\beta_o+w$中恢复向量$\beta_o的问题，其中$X$是测量矩阵，$w$是噪声。套索估计由优化问题的解$HAT{\beta}_{\lambda}=\arg\min_{\beta}\frac{1}{2}y-X\beta_2^2+\lambda\\beta_1$给出。在已提出的求解该优化问题的迭代算法中，近似消息传递算法(AMP)以其快速收敛而备受关注。尽管LASSO和AMP估计的理论分析取得了很大进展，但对它们作为正则化参数或剩余参数的函数的行为却知之甚少。例如，以下基本问题在文献中尚未被研究：(I)活动集的大小如何表现为$\lambda$的函数？(Ii)均方误差如何表现为$\lambda$的函数？(Iii)$beta^t-beta_o_2^2/p$如何表现为$tau^1，1dots，tau^{t-1}$？回答这些问题将有助于解决有关$\lambda$或$\tau^1，\tau^2，\ldots$的优化的实际挑战。本文在渐近的情况下回答了这些问题，并说明了如何利用这些结果来推导出调整AMP的参数或套索的参数的简单且理论上最优的方法。探讨了AMP参数的最佳调谐与套索的最佳调谐之间的关系。
We consider the problem of recovering a vector $\beta_o \in \mathbb{R}^p$ from $n$ random and noisy linear observations $y= X\beta_o + w$, where $X$ is the measurement matrix and $w$ is noise. The LASSO estimate is given by the solution to the optimization problem $\hat{\beta}_{\lambda} = \arg \min_{\beta} \frac{1}{2} \|y-X\beta\|_2^2 + \lambda \| \beta \|_1$. Among the iterative algorithms that have been proposed for solving this optimization problem, approximate message passing (AMP) has attracted attention for its fast convergence. Despite significant progress in the theoretical analysis of the estimates of LASSO and AMP, little is known about their behavior as a function of the regularization parameter $\lambda$, or the thereshold parameters $\tau^t$. For instance the following basic questions have not yet been studied in the literature: (i) How does the size of the active set $\|\hat{\beta}^\lambda\|_0/p$ behave as a function of $\lambda$? (ii) How does the mean square error $\|\hat{\beta}_{\lambda} - \beta_o\|_2^2/p$ behave as a function of $\lambda$? (iii) How does $\|\beta^t - \beta_o \|_2^2/p$ behave as a function of $\tau^1, \ldots, \tau^{t-1}$? Answering these questions will help in addressing practical challenges regarding the optimal tuning of $\lambda$ or $\tau^1, \tau^2, \ldots$. This paper answers these questions in the asymptotic setting and shows how these results can be employed in deriving simple and theoretically optimal approaches for tuning the parameters $\tau^1, \ldots, \tau^t$ for AMP or $\lambda$ for LASSO. It also explores the connection between the optimal tuning of the parameters of AMP and the optimal tuning of LASSO.
影响因子：
1.6
作者：
Amelunxen, Dennis;Lotz, Martin;Tropp, Joel A.
通讯作者：
Tropp, Joel A.
影响因子：
4.5
作者：
Lockhart R;Taylor J;Tibshirani RJ;Tibshirani R
通讯作者：
Tibshirani R