Self-Paced Learning for Latent Variable Models
Self-Paced Learning for Latent Variable Models
复制标题
DOI：
--
复制
发表时间：
2010-12
期刊：
影响因子：
--
通讯作者：
M. P. Kumar;Ben Packer;D. Koller
中科院分区：
文献类型：
--
作者：
M. P. Kumar;Ben Packer;D. Koller
潜在变量模型是解决机器学习中若干任务的强大工具。然而，隐变量模型参数学习算法容易陷入糟糕的局部最优。为了缓解这个问题，我们建立在直觉的基础上，而不是同时考虑所有样本，算法应该以有意义的顺序呈现训练数据，以促进学习。样品的顺序取决于它们的难易程度。主要的挑战是，我们经常没有提供一个容易计算的测量样本的容易程度。我们通过提出一种新颖的迭代自定节奏学习算法来解决这个问题，其中每次迭代同时选择简单的样本并学习新的参数向量。所选样本的数量由一个权重控制，该权重被退火，直到考虑了整个训练数据。我们的经验证明，自定节奏学习算法在四个应用上优于当前的学习潜在结构支持向量机的方法：对象定位、名词短语共指、motif查找和手写数字识别。
Latent variable models are a powerful tool for addressing several tasks in machine learning. However, the algorithms for learning the parameters of latent variable models are prone to getting stuck in a bad local optimum. To alleviate this problem, we build on the intuition that, rather than considering all samples simultaneously, the algorithm should be presented with the training data in a meaningful order that facilitates learning. The order of the samples is determined by how easy they are. The main challenge is that often we are not provided with a readily computable measure of the easiness of samples. We address this issue by proposing a novel, iterative self-paced learning algorithm where each iteration simultaneously selects easy samples and learns a new parameter vector. The number of samples selected is governed by a weight that is annealed until the entire training data has been considered. We empirically demonstrate that the self-paced learning algorithm outperforms the state of the art method for learning a latent structural SVM on four applications: object localization, noun phrase coreference, motif finding and handwritten digit recognition.