Accuracy of latent-variable estimation in Bayesian semi-supervised learning
Accuracy of latent-variable estimation in Bayesian semi-supervised learning
复制标题
DOI：
10.1016/j.neunet.2015.04.012
复制
发表时间：
2013-08
期刊：
影响因子：
--
通讯作者：
Keisuke Yamazaki
中科院分区：
文献类型：
--
作者：
Keisuke Yamazaki
分层概率模型，如高斯混合模型，被广泛用于无监督学习任务。这些模型由可观测变量和潜在变量组成，它们分别代表可观测数据和潜在数据生成过程。无监督学习任务，如聚类分析，被认为是基于可观测变量对潜在变量的估计。在半监督学习中，当观察到一些标签时，潜在变量的估计将比无监督学习更精确，其中一个令人关注的问题是澄清标签数据的影响。然而，对于潜变量估计的准确性，目前还没有足够的理论分析。在以前的研究中，建立了一个基于分布的误差函数，并计算了其渐近形式，用于带有产生式模型的无监督学习。结果表明，对于潜在变量的估计，贝叶斯方法比最大似然方法更准确。给出了判别模型和产生式模型下贝叶斯半监督学习误差函数的渐近形式。结果表明，当模型被很好地指定时，使用所有给定数据的生成模型的性能更好。
Hierarchical probabilistic models, such as Gaussian mixture models, are widely used for unsupervised learning tasks. These models consist of observable and latent variables, which represent the observable data and the underlying data-generation process, respectively. Unsupervised learning tasks, such as cluster analysis, are regarded as estimations of latent variables based on the observable ones. The estimation of latent variables in semi-supervised learning, where some labels are observed, will be more precise than that in unsupervised, and one of the concerns is to clarify the effect of the labeled data. However, there has not been sufficient theoretical analysis of the accuracy of the estimation of latent variables. In a previous study, a distribution-based error function was formulated, and its asymptotic form was calculated for unsupervised learning with generative models. It has been shown that, for the estimation of latent variables, the Bayes method is more accurate than the maximum-likelihood method. The present paper reveals the asymptotic forms of the error function in Bayesian semi-supervised learning for both discriminative and generative models. The results show that the generative model, which uses all of the given data, performs better when the model is well specified.