Comment: Regularization via Bayesian Penalty Mixing
Comment: Regularization via Bayesian Penalty Mixing
复制标题
DOI：
10.1080/00401706.2020.1801258
复制
发表时间：
2020-10
期刊：
影响因子：
2.5
通讯作者：
E. George;V. Ročková
中科院分区：
文献类型：
--
作者：
E. George;V. Ročková
岭回归和套索说明了惩罚似然正则化在回归分析中的一些显著成功。当通过后验最大化的贝叶斯透镜观察时，它们的惩罚函数的正则化效应可以理解为源于回归系数上的先验分布。这一著名的贝叶斯观点进一步揭示了由这种罚函数引起的收缩的本质，并为从先验空间中的概率混合创建新的罚函数打开了大门。这里用扣球和板式套索说明了这种贝叶斯判罚混合创造点球的潜力。尖峰套索和板条套索是套索估值器的自适应凸组合，它自动对小系数采用强阈值收缩，对大系数自动采用弱稳定收缩，被认为提供了同时的变量选择和对选定系数的几乎无偏的估计。
Ridge regression and the lasso illustrate some of the remarkable successes of penalized likelihood regularization for regression analysis. When viewed through the Bayesian lens of posterior maximization, the regularizing effects of their penalty functions can be understood as stemming from prior distributions over the regression coefficients. This well-known Bayesian perspective provides further insights into the nature of the shrinkage induced by such penalty functions, and opens the door for the creation of new penalty functions from probabilistic mixtures in the space of priors. The potential of such Bayesian penalty mixing for penalty creation is here illustrated with the spike and slab lasso prior. An adaptive convex combination of lasso estimators which automatically employ strong thresholding shrinkage to small coefficients and weak stabilizing shrinkage to large coefficients, the spike and slab lasso is seen to offer simultaneous variable selection and nearly unbiased estimation of the selected coefficients.