Bregman divergence as general framework to estimate unnormalized statistical models
Bregman divergence as general framework to estimate unnormalized statistical models
复制标题
DOI：
--
复制
发表时间：
2011-07
期刊：
影响因子：
--
通讯作者：
Michael U Gutmann;J. Hirayama
中科院分区：
文献类型：
--
作者：
Michael U Gutmann;J. Hirayama
我们表明Bregman散度提供了一个丰富的框架来估计连续或离散随机变量的非归一化统计模型，即分别不积分或求和为1的模型。我们证明了最近的估计方法，如噪声对比估计、比率匹配和分数匹配属于所提出的框架，并解释了它们基于监督学习的相互联系。进一步，我们讨论了助推在无监督学习中的作用。
We show that the Bregman divergence provides a rich framework to estimate unnormalized statistical models for continuous or discrete random variables, that is, models which do not integrate or sum to one, respectively. We prove that recent estimation methods such as noise-contrastive estimation, ratio matching, and score matching belong to the proposed framework, and explain their interconnection based on supervised learning. Further, we discuss the role of boosting in unsupervised learning.