SURE-tuned bridge regression
SURE-tuned bridge regression
复制标题
DOI：
10.1007/s11222-023-10350-z
复制
发表时间：
2024-02-01
影响因子：
2.2
通讯作者：
Bhadra，Anindya
中科院分区：
文献类型：
--
作者：
Loria，Jorge;Bhadra，Anindya
考虑正则化线性回归，也称为桥回归。因为，桥式回归具有一些有趣的统计特性，如估计的稀疏性和近无偏性（Fan和Li in J Am Stat Assoc 96(456): 1348 - 1360,2001）。然而，主要的困难在于这些值的惩罚的非凸性，这使得优化过程具有挑战性，并且通常只能找到局部最优。为了解决这个问题，Polson等人（J R Stat Soc B 76(4): 713-733, 2013）采用基于抽样的全贝叶斯方法来解决这个问题，使用桥罚和回归系数的幂指数先验之间的对应关系。然而，他们的采样过程依赖于马尔可夫链蒙特卡罗（MCMC）技术，该技术本身是顺序的，不能扩展到大的问题维度。交叉验证方法同样是计算密集型的。为此，我们的贡献是一种新的非迭代方法来拟合桥回归模型。主要贡献在于Stein对桥式回归的样本外预测风险的无偏风险估计的显式公式，然后可以优化以选择所需的调优参数，使我们能够完全绕过MCMC以及计算密集型交叉验证方法。与迭代方案相比，我们的过程产生的计算时间只有一小部分，而统计性能没有任何明显的损失。anrimplentation可以在网上公开获取：https://github.com/loriaJ/Sure-tuned_BridgeRegression。
Consider theregularized linear regression, also termed Bridge regression. For, Bridge regression enjoys several statistical properties of interest such as sparsity and near-unbiasedness of the estimates (Fan and Li in J Am Stat Assoc 96(456): 1348–1360, 2001). However, the main difficulty lies in the non-convex nature of the penalty for these values of, which makes an optimization procedure challenging and usually it is only possible to find a local optimum. To address this issue, Polson et al. (J R Stat Soc B 76(4):713–733, 2013) took a sampling based fully Bayesian approach to this problem, using the correspondence between the Bridge penalty and a power exponential prior on the regression coefficients. However, their sampling procedure relies on Markov chain Monte Carlo (MCMC) techniques, which are inherently sequential and not scalable to large problem dimensions. Cross validation approaches are similarly computation-intensive. To this end, our contribution is a novelnon-iterativemethod to fit a Bridge regression model. The main contribution lies in an explicit formula for Stein’s unbiased risk estimate for the out of sample prediction risk of Bridge regression, which can then be optimized to select the desired tuning parameters, allowing us to completely bypass MCMC as well as computation-intensive cross validation approaches. Our procedure yields results in a fraction of computational times compared to iterative schemes, without any appreciable loss in statistical performance. AnRimplementation is publicly available online at: https://github.com/loriaJ/Sure-tuned_BridgeRegression.