A uniform framework for the combination of penalties in generalized structured models
A uniform framework for the combination of penalties in generalized structured models
复制标题
DOI：
10.1007/s11634-015-0205-y
复制
发表时间：
2017-03-01
影响因子：
1.6
通讯作者：
Tutz, Gerhard
中科院分区：
文献类型：
--
作者：
Oelker, Margret-Ruth;Tutz, Gerhard
惩罚估计已经成为回归模型中正则化和模型选择的一个成熟工具。各种各样的惩罚与特定的功能是可用的，并提出了有效的算法，具体的处罚。但是，没有太多的资料可以用来拟合具有不同惩罚组合的模型。当在我们的应用程序中对慕尼黑的租金数据进行建模时，各种类型的预测器需要在一个模型中组合Ridge、group Lasso和Lasso类型的惩罚。我们建议近似罚款是（半）规范的标量线性变换的系数向量在广义结构化模型，使各种处罚可以结合在一个模型。该方法是非常通用的，例如Lasso，融合Lasso，Ridge，平滑剪切绝对偏差惩罚，弹性网络和更多的惩罚被嵌入。计算是基于传统的惩罚迭代加权最小二乘算法，因此，易于实现。新的惩罚措施可以很快纳入。该方法扩展到基于向量的参数的处罚。有几种选择惩罚参数的可能性。软件实现可用。一些说明性的例子显示了有希望的结果。
Penalized estimation has become an established tool for regularization and model selection in regression models. A variety of penalties with specific features are available and effective algorithms for specific penalties have been proposed. But not much is available to fit models with a combination of different penalties. When modeling the rent data of Munich as in our application, various types of predictors call for a combination of a Ridge, a group Lasso and a Lasso-type penalty within one model. We propose to approximate penalties that are (semi-)norms of scalar linear transformations of the coefficient vector in generalized structured models-such that penalties of various kinds can be combined in one model. The approach is very general such that the Lasso, the fused Lasso, the Ridge, the smoothly clipped absolute deviation penalty, the elastic net and many more penalties are embedded. The computation is based on conventional penalized iteratively re-weighted least squares algorithms and hence, easy to implement. New penalties can be incorporated quickly. The approach is extended to penalties with vector based arguments. There are several possibilities to choose the penalty parameter(s). A software implementation is available. Some illustrative examples show promising results.