Minimax Optimal Estimation of KL Divergence for Continuous Distributions
Minimax Optimal Estimation of KL Divergence for Continuous Distributions
复制标题
DOI：
10.1109/tit.2020.3009923
复制
发表时间：
2020-02
影响因子：
2.5
通讯作者：
Puning Zhao;L. Lai
中科院分区：
文献类型：
--
作者：
Puning Zhao;L. Lai
从完全相同且独立分布的样本中估计Kullback-Leibler散度是一个重要的问题。一个简单而有效的估计是基于这些样本之间的$k$最近邻距离。在本文中，我们分析了这种估计的偏差和方差的收敛速度。此外，我们推导出了极大极小均方误差的下界，并证明了kNN方法是渐近速率最优的。
Estimating Kullback-Leibler divergence from identical and independently distributed samples is an important problem in various domains. One simple and effective estimator is based on the $k$ nearest neighbor distances between these samples. In this paper, we analyze the convergence rates of the bias and variance of this estimator. Furthermore, we derive a lower bound of the minimax mean square error and show that kNN method is asymptotically rate optimal.