Unified Perspective on Probability Divergence via the Density-Ratio Likelihood: Bridging KL-Divergence and Integral Probability Metrics
Unified Perspective on Probability Divergence via the Density-Ratio Likelihood: Bridging KL-Divergence and Integral Probability Metrics
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Masahiro Kato;M. Imaizumi;Kentaro Minami
中科院分区：
文献类型：
--
作者：
Masahiro Kato;M. Imaizumi;Kentaro Minami
本文从最大似然密度比估计(DRE)的角度对Kullback-Leibler(KL)散度和积分概率度量(IPM)提供了一个统一的视角。KL-散度和综合决策支持向量机在创成式建模等fi领域有着广泛的应用。然而，对这些概念的统一理解仍未得到探索。本文证明了KL-散度和IPMS可以表示为仅因抽样方案不同而不同的极大似然，并利用这一结果导出了IPMS的统一的fi形式和一种松弛估计方法。为了解决这一估计问题，我们构造了一个无约束极大似然估计器，用于在分层抽样方案下进行DRE。我们进一步提出了一类新的概率散度，称为密度比度量(DRM)，它内插了KL-散度和IPMS。除了这些fi编码外，我们还介绍了DRM的一些应用，如DRE和生成式对抗网络。在实验中，我们验证了我们提出的方法的有效性。
This paper provides a unified perspective for the Kullback-Leibler (KL)-divergence and the integral probability metrics (IPMs) from the perspective of maximum likelihood density-ratio estimation (DRE). Both the KL-divergence and the IPMs are widely used in various fields in applications such as generative modeling. However, a unified understanding of these concepts has still been unexplored. In this paper, we show that the KL-divergence and the IPMs can be represented as maximal likelihoods differing only by sampling schemes, and use this result to derive a unified form of the IPMs and a relaxed estimation method. To develop the estimation problem, we construct an unconstrained maximum likelihood estimator to perform DRE with a stratified sampling scheme. We further propose a novel class of probability divergences, called the Density Ratio Metrics (DRMs), that interpolates the KL-divergence and the IPMs. In addition to these findings, we also introduce some applications of the DRMs, such as DRE and generative adversarial networks. In experiments, we validate the effectiveness of our proposed methods.