Lifted Online Training of Relational Models with Stochastic Gradient Methods
Lifted Online Training of Relational Models with Stochastic Gradient Methods
复制标题
用随机梯度方法提升关系模型的在线训练
DOI：
10.1007/978-3-642-33460-3_43
复制
发表时间：
2012
影响因子：
22.7
通讯作者：
Sriraam Natarajan
中科院分区：
文献类型：
--
作者：
B. Ahmadi;K. Kersting;Sriraam Natarajan
提升推理方法通过使用对称性来处理整组不可区分的随机变量，使得以前难以处理的大型概率推理问题可以快速解决。尽管如此，在许多情况下，如果不是大多数情况下，训练关系模型将不会从提升中受益：模型中的对称性很容易被打破，因为变量由于不对称地依赖于证据而变得相关。对于这种情况，一个吸引人的想法是培训和重组当地的模式。这打破了长期的依赖，并允许在本地培训任务内和跨本地培训任务利用提升。此外，它自然为关系模型的在线培训铺平了道路。具体地说，我们发展了第一种具有增益向量自适应的提升随机梯度优化方法，它逐一处理提升的每一块。在几个数据集上，最终的优化器收敛到相同质量的解决方案的速度要快一个数量级，这只是因为与批处理训练不同，它在看到整个巨型示例之前很久就开始优化了。
Lifted inference approaches have rendered large, previously intractable probabilistic inference problems quickly solvable by employing symmetries to handle whole sets of indistinguishable random variables. Still, in many if not most situations training relational models will not benefit from lifting: symmetries within models easily break since variables become correlated by virtue of depending asymmetrically on evidence. An appealing idea for such situations is to train and recombine local models. This breaks long-range dependencies and allows to exploit lifting within and across the local training tasks. Moreover, it naturally paves the way for online training for relational models. Specifically, we develop the first lifted stochastic gradient optimization method with gain vector adaptation, which processes each lifted piece one after the other. On several datasets, the resulting optimizer converges to the same quality solution over an order of magnitude faster, simply because unlike batch training it starts optimizing long before having seen the entire mega-example even once.