Adaptive sparseness for supervised learning
Adaptive sparseness for supervised learning
复制标题
DOI：
10.1109/tpami.2003.1227989
复制
发表时间：
2003-09-01
影响因子：
23.6
通讯作者：
Figueiredo, MAT
中科院分区：
文献类型：
--
作者：
Figueiredo, MAT
监督团队的目标是根据一组训练示例推断出功能映射。为了实现良好的泛化，有必要控制学习函数的“复杂度”。在贝叶斯方法中，这是通过调整被组合函数的参数的先验来完成的。我们提出了一种贝叶斯方法来进行监督团队，它会导致稀疏解；也就是说，其中不相关的参数被自动精确地设置为零。其他获得稀疏分类器（如拉普拉斯先验，支持向量机）的方法涉及（超）参数，这些参数控制结果分类器的稀疏程度；这些参数必须以某种方式根据训练数据进行调整/估计。相反，我们的方法不涉及任何需要调整或估计的（超）参数。这是通过对拉普拉斯先验的层次-贝叶斯解释来实现的，然后通过采用杰弗里斯的非信息超先验来修改。通过期望最大化（EM）算法实现。几个基准数据集的实验表明，所提出的方法产生了最先进的性能。特别是，我们的方法优于支持向量机，并与最佳替代技术竞争，尽管它不涉及对稀疏控制超参数的调优或调整。
The goal of supervised teaming is to infer a functional mapping based on a set of training examples. To achieve good generalization, it is necessary to control the "complexity" of the learned function. In Bayesian approaches, this is done by adapting a prior for the parameters of the function being teamed. We propose a Bayesian approach to supervised teaming, which leads to sparse solutions; that is, in which irrelevant parameters are automatically set exactly to zero. Other ways to obtain sparse classifiers (such as Laplacian priors, support vector machines) involve (hyper)parameters which control the degree of sparseness of the resulting classifiers; these parameters have to be somehow adjusted/estimated from the training data. In contrast, our approach does not involve any (hyper)parameters to be adjusted or estimated. This is achieved by a hierarchical-Bayes interpretation of the Laplacian prior, which is then modified by the adoption of a Jeffreys' noninformative hyperprior. Implementation is carried out by an expectation-maximization (EM) algorithm. Experiments with several benchmark data sets show that the proposed approach yields state-of-the-art performance. In particular, our method outperforms SVMs and performs competitively with the best alternative techniques, although it involves no tuning or adjustment of sparseness-controlling hyperparameters.