Scalable Graph-Based Semi-Supervised Learning through Sparse Bayesian Model
Scalable Graph-Based Semi-Supervised Learning through Sparse Bayesian Model
复制标题
通过稀疏贝叶斯模型的可扩展的基于图的半监督学习
DOI：
10.1109/tkde.2017.2749574
复制
发表时间：
2017-12-01
影响因子：
8.9
通讯作者：
Yao, Xin
中科院分区：
文献类型：
--
作者：
Jiang, Bingbing;Chen, Huanhuan;Yao, Xin
半监督学习研究的是如何利用未标记数据的先验知识来提高分类器的性能。近年来，基于流形假设或聚类假设的SSL方法被广泛应用于分类器中。特别是，基于图的方法，以下的流形假设，已经取得了很好的性能在许多现实世界中的应用。然而，它们中的大多数仅在小规模数据集上运行良好，并且缺乏概率输出。通过定义基于图的稀疏先验，提出了一种基于稀疏贝叶斯模型的可扩展图SSL框架。在传统贝叶斯推理技术的基础上，得到了一种稀疏贝叶斯SSL算法（（SBSL）-L-2），该算法能够去除无关的未标记样本，并对样本外数据进行概率预测。此外，为了将（SBSL）-L-2扩展到大规模数据集，导出了增量（SBSL）-L-2（（ISBSL）-L-2）。（ISBSL）-L-2的核心思想是采用增量策略，顺序选择对学习有贡献的部分未标记样本，而不是直接使用所有可用的未标记样本。（ISBSL）-L-2具有较低的时间和空间复杂度比以前的SSL算法使用所有未标记的样本。在不同数据集上的实验表明，该算法可以达到相当的分类效果和效率，具有更好的可扩展性。最后，基于鲁棒性分析，给出了推广误差界。
Semi-supervised learning (SSL) concerns the problem of how to improve classifiers' performance through making use of prior knowledge from unlabeled data. Many SSL methods have been developed to integrate unlabeled data into the classifiers based on either the manifold or cluster assumption in recent years. In particular, the graph-based approaches, following the manifold assumption, have achieved a promising performance in many real-world applications. However, most of them work well on small-scale data sets only and lack probabilistic outputs. In this paper, a scalable graph-based SSL framework through sparse Bayesian model is proposed by defining a graph-based sparse prior. Based on the traditional Bayesian inference technique, a sparse Bayesian SSL algorithm ((SBSL)-L-2) is obtained, which can remove the irrelevant unlabeled samples and make probabilistic prediction for out-of-sample data. Moreover, in order to scale (SBSL)-L-2 to large-scale data sets, an incremental (SBSL)-L-2 ((ISBSL)-L-2) is derived. The key idea of (ISBSL)-L-2 is employing an incremental strategy and sequentially selecting parts of unlabeled samples that contribute to the learning instead of using all available unlabeled samples directly. (ISBSL)-L-2 has lower time and space complexities than previous SSL algorithms with the use of all unlabeled samples. Extensive experiments on various data sets verify that our algorithms can achieve comparable classification effectiveness and efficiency with much better scalability. Finally, the generalization error bound is derived based on robustness analysis.