The Horseshoe-Like Regularization for Feature Subset Selection
The Horseshoe-Like Regularization for Feature Subset Selection
复制标题
用于特征子集选择的马蹄形正则化
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Brandon T. Willard
中科院分区：
文献类型：
--
作者：
A. Bhadra;J. Datta;Nicholas G. Polson;Brandon T. Willard
特征子集选择出现在统计学的许多高维应用中，例如压缩传感和基因组学。对于这项任务，ℓ0惩罚是理想的，但需要注意的是，它需要对所有模型进行NP-Hard组合评估。最近一个相当感兴趣的领域是开发有效的算法来用ℓγ(0，1)的非凸γ∈惩罚来拟合模型，这导致模型比凸ℓ1或套索惩罚更稀疏，但更难拟合。我们提出了一种新的特征子集选择方法--马蹄形正则化惩罚，并证明了它的理论和计算优势。与现有的非凸优化方法不同的是，惩罚的完全概率表示为适当先验的对数的负值，这反过来又使得优化的期望最大化和局部线性逼近算法以及不确定性量化的MCMC算法成为可能。在合成和实际数据中，所得到的算法提供了更好的统计性能，并且计算所需的时间仅为最先进的非凸解算器的一小部分。
Feature subset selection arises in many high-dimensional applications of statistics, such as compressed sensing and genomics. The ℓ0 penalty is ideal for this task, the caveat being it requires the NP-hard combinatorial evaluation of all models. A recent area of considerable interest is to develop efficient algorithms to fit models with a non-convex ℓγ penalty for γ ∈ (0,1), which results in sparser models than the convex ℓ1 or lasso penalty, but is harder to fit. We propose an alternative, termed the horseshoe regularization penalty for feature subset selection, and demonstrate its theoretical and computational advantages. The distinguishing feature from existing non-convex optimization approaches is a full probabilistic representation of the penalty as the negative of the logarithm of a suitable prior, which in turn enables efficient expectation-maximization and local linear approximation algorithms for optimization and MCMC for uncertainty quantification. In synthetic and real data, the resulting algorithms provide better statistical performance, and the computation requires a fraction of time of state-of-the-art non-convex solvers.
影响因子：
4.5
作者：
Hunter, DR;Li, RZ
通讯作者：
Li, RZ
影响因子：
2.7
作者：
Bhattacharya A;Chakraborty A;Mallick BK
通讯作者：
Mallick BK