Asymptotic Bayesian Generalization Error in Latent Dirichlet Allocation and Stochastic Matrix Factorization.
Asymptotic Bayesian Generalization Error in Latent Dirichlet Allocation and Stochastic Matrix Factorization.
复制标题
潜在狄利克雷分配和随机矩阵分解中的渐近贝叶斯泛化误差。
DOI：
10.1007/s42979-020-0071-3
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Sumio Watanabe.
中科院分区：
文献类型：
--
作者：
Naoki Hayashi;Sumio Watanabe.
潜在狄利克雷分配（LDA）在文档分析、图像处理和许多信息系统中是有用的，然而，由于它是一个奇异的学习机器，常规的统计理论不能应用于它，因此它的泛化性能一直是未知的。随机矩阵分解（英语：Stochastic matrix factorization，SMF）是一种限制矩阵分解，其中矩阵因子是随机的;矩阵的列在单纯形中。SMF被应用于图像识别和文本挖掘。我们可以将SMF理解为一个统计模型，通过该模型，给定数据的随机矩阵由两个随机矩阵的乘积表示，由于非正则性，其泛化性能也是未知的。本文利用代数和几何方法证明了LDA和SMF的解析等价性，它们具有相同的真实的对数正则阈值（RLCT），从而使它们具有相同的贝叶斯推广误差和相同的对数边际似然。此外，我们还导出了RLCT的上界，并证明了它小于参数的维数除以2，因此它们的贝叶斯推广误差小于常规统计模型的贝叶斯推广误差。
Latent Dirichlet allocation (LDA) is useful in document analysis, image processing, and many information systems; however, its generalization performance has been left unknown because it is a singular learning machine to which regular statistical theory can not be applied. Stochastic matrix factorization (SMF) is a restricted matrix factorization in which matrix factors are stochastic; the column of the matrix is in a simplex. SMF is being applied to image recognition and text mining. We can understand SMF as a statistical model by which a stochastic matrix of given data is represented by a product of two stochastic matrices, whose generalization performance has also been left unknown because of non-regularity. In this paper, using an algebraic and geometric method, we show the analytic equivalence of LDA and SMF, both of which have the same real log canonical threshold (RLCT), resulting in that they asymptotically have the same Bayesian generalization error and the same log marginal likelihood. Moreover, we derive the upper bound of the RLCT and prove that it is smaller than the dimension of the parameter divided by two, hence the Bayesian generalization errors of them are smaller than those of regular statistical models.