A Unified Statistically Efficient Estimation Framework for Unnormalized Models
A Unified Statistically Efficient Estimation Framework for Unnormalized Models
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Masatoshi Uehara;T. Kanamori;Takashi Takenouchi;T. Matsuda
中科院分区：
文献类型：
--
作者：
Masatoshi Uehara;T. Kanamori;Takashi Takenouchi;T. Matsuda
非规范化模型的参数估计是一个具有挑战性的问题。最大似然估计（MLE）的计算是不可行的，因为这些模型的归一化常数没有明确计算。虽然一些一致的估计已经提出了较早，统计效率的问题仍然存在。在这项研究中，我们提出了一个统一的，统计上有效的估计框架的非正规化模型和几个有效的估计，其渐近方差是相同的极大似然估计。这些估计量的计算成本也是合理的，无论样本空间是离散的还是连续的，它们都可以使用。通过结合以下两种方法推导出所提出的估计的损失函数：（1）使用Bregman散度的密度比匹配，和（2）插入非参数估计。我们还分析了所提出的估计的性质时，未规范化的模型是错误的。实验结果表明，我们的方法优于现有的方法。
The parameter estimation of unnormalized models is a challenging problem. The maximum likelihood estimation (MLE) is computationally infeasible for these models since normalizing constants are not explicitly calculated. Although some consistent estimators have been proposed earlier, the problem of statistical efficiency remains. In this study, we propose a unified, statistically efficient estimation framework for unnormalized models and several efficient estimators, whose asymptotic variance is the same as the MLE. The computational cost of these estimators is also reasonable and they can be employed whether the sample space is discrete or continuous. The loss functions of the proposed estimators are derived by combining the following two methods: (1) density-ratio matching using Bregman divergence, and (2) plugging-in nonparametric estimators. We also analyze the properties of the proposed estimators when the unnormalized models are misspecified. The experimental results demonstrate the advantages of our method over existing approaches.