Estimation of the entropy based on its polynomial representation.
Estimation of the entropy based on its polynomial representation.
复制标题
DOI：
10.1103/physreve.85.051139
复制
发表时间：
2012-05
期刊：
影响因子：
--
通讯作者：
M. Vinck;F. Battaglia;V. Balakirsky;A. Vinck;C. Pennartz
中科院分区：
文献类型：
--
作者：
M. Vinck;F. Battaglia;V. Balakirsky;A. Vinck;C. Pennartz
从有限大小的经验样本中估算熵对于信息理论以及复杂统计系统的分析至关重要。然而，这项微妙的任务受到内在统计偏见的破坏。在这里，我们将熵函数分解为多项式近似函数和剩余函数。近似函数基于对数的泰勒膨胀。考虑到n的观察，我们根据K巧克力的计数集对第一n个功率序列项进行了公正的线性估计。对于剩余功能，我们使用非线性贝叶斯估计值，其熵在恩门曼，沙菲和比亚莱克开发的熵上几乎平坦。我们的模拟表明，与其他可用估计器相比，组合的熵估计器降低了偏差。
Estimating entropy from empirical samples of finite size is of central importance for information theory as well as the analysis of complex statistical systems. Yet, this delicate task is marred by intrinsic statistical bias. Here we decompose the entropy function into a polynomial approximation function and a remainder function. The approximation function is based on a Taylor expansion of the logarithm. Given n observations, we give an unbiased, linear estimate of the first n power series terms based on counting sets of k coincidences. For the remainder function we use nonlinear Bayesian estimation with a nearly flat prior distribution on the entropy that was developed by Nemenman, Shafee, and Bialek. Our simulations show that the combined entropy estimator has reduced bias in comparison to other available estimators.