Incremental Feature Selection and l1 Regularization for Relaxed Maximum-Entropy Modeling
Incremental Feature Selection and l1 Regularization for Relaxed Maximum-Entropy Modeling
复制标题
松弛最大熵建模的增量特征选择和 l1 正则化
DOI：
--
复制
发表时间：
2004
期刊：
影响因子：
--
通讯作者：
A. Vasserman
中科院分区：
文献类型：
--
作者：
S. Riezler;A. Vasserman
对于对数线性模型的似然最大化，我们提出了一种有界约束松弛的方法，该方法对应于在似然最大化中使用双指数先验或`1正则化。Perkins等人提出了一种增量特征选择和正则化相结合的方法，通过将正则化方法自然地结合到基于梯度的特征选择中，可以建立用于最大熵建模的增量特征选择方法。(2003)。这为全特征集上的标准正则化提供了一种有效的替代方案，并为基于似然的特征选择中使用的阈值技术提供了数学上的合理性。此外，我们还对具有中等冗余度的语言特征集的n-Best特征选择进行了扩展，并给出了实验结果，实验结果表明，对于最大熵分析任务，该方法优于`0，1-Best`1，`2正则化和标准增量特征选择。1
We present an approach to bounded constraintrelaxation for entropy maximization that corresponds to using a double-exponential prior or `1 regularizer in likelihood maximization for log-linear models. We show that a combined incremental feature selection and regularization method can be established for maximum entropy modeling by a natural incorporation of the regularizer into gradientbased feature selection, following Perkins et al. (2003). This provides an efficient alternative to standard `1 regularization on the full feature set, and a mathematical justification for thresholding techniques used in likelihood-based feature selection. Also, we motivate an extension to n-best feature selection for linguistic features sets with moderate redundancy, and present experimental results showing its advantage over `0, 1-best `1, `2 regularization and over standard incremental feature selection for the task of maximum-entropy parsing.1