Bayesian estimation of the Kullback-Leibler divergence for categorical systems using mixtures of Dirichlet priors
Bayesian estimation of the Kullback-Leibler divergence for categorical systems using mixtures of Dirichlet priors
复制标题
使用混合狄利克雷先验对分类系统的 Kullback-Leibler 散度进行贝叶斯估计
DOI：
10.1103/physreve.109.024305
复制
发表时间：
2024
影响因子：
2.4
通讯作者：
Walczak, Aleksandra M.
中科院分区：
文献类型：
--
作者：
Camaglia, Francesco;Nemenman, Ilya;Mora, Thierry;Walczak, Aleksandra M.
在生物学、工程和经济学的许多应用中，识别复杂过程的数据分布之间的相似性和差异需要比较离散计数的有限分类样本。统计差异量化了两个分布之间的差异。然而，它们的估计非常困难，经验方法常常失败，特别是当样本较小时。我们开发了两个概率分布之间的 Kullback-Leibler 散度的贝叶斯估计器，该估计器利用了正在比较的分布的狄利克雷先验的混合。我们通过两个例子研究估计器的属性：从狄利克雷分布中提取的概率和从马尔可夫链中提取的随机字母串。我们将该方法扩展到平方 Hellinger 散度。这两种估计器都优于其他估计技术，对于具有大量类别的数据和较高的散度值，具有更好的结果。
In many applications in biology, engineering, and economics, identifying similarities and differences between distributions of data from complex processes requires comparing finite categorical samples of discrete counts. Statistical divergences quantify the difference between two distributions. However, their estimation is very difficult and empirical methods often fail, especially when the samples are small. We develop a Bayesian estimator of the Kullback-Leibler divergence between two probability distributions that makes use of a mixture of Dirichlet priors on the distributions being compared. We study the properties of the estimator on two examples: probabilities drawn from Dirichlet distributions and random strings of letters drawn from Markov chains. We extend the approach to the squared Hellinger divergence. Both estimators outperform other estimation techniques, with better results for data with a large number of categories and for higher values of divergences.