Approximations of Shannon Mutual Information for Discrete Variables with Applications to Neural Population Coding.
Approximations of Shannon Mutual Information for Discrete Variables with Applications to Neural Population Coding.
复制标题
DOI：
10.3390/e21030243
复制
发表时间：
2019-03-04
期刊：
影响因子：
--
通讯作者：
Zhang K
中科院分区：
文献类型：
--
作者：
Huang W;Zhang K
虽然香农互信息已经得到了广泛的应用，但它的有效计算在许多实际问题中往往是困难的，包括在神经种群编码中的问题。基于Fisher信息的渐近公式有时提供对互信息的精确近似，但这种方法仅限于连续变量，因为Fisher信息的计算需要关于编码变量的导数。在这篇文章中，我们考虑了基于Kullback-Leibler散度和Rényi散度的互信息的信息论界和近似。在神经种群编码的背景下，我们提出了几种信息度量来近似香农互信息。虽然我们的渐近公式都适用于离散变量，但其中一个公式无论编码变量是离散的还是连续的，都具有一致的性能和高的精度。我们进行了数值模拟，并证实了我们的近似公式对于近似大神经元群体的刺激和响应之间的互信息具有很高的精度。这些近似公式可能会给信息论在许多实际和理论问题上的应用带来便利。
Although Shannon mutual information has been widely used, its effective calculation is often difficult for many practical problems, including those in neural population coding. Asymptotic formulas based on Fisher information sometimes provide accurate approximations to the mutual information but this approach is restricted to continuous variables because the calculation of Fisher information requires derivatives with respect to the encoded variables. In this paper, we consider information-theoretic bounds and approximations of the mutual information based on Kullback-Leibler divergence and Rényi divergence. We propose several information metrics to approximate Shannon mutual information in the context of neural population coding. While our asymptotic formulas all work for discrete variables, one of them has consistent performance and high accuracy regardless of whether the encoded variables are discrete or continuous. We performed numerical simulations and confirmed that our approximation formulas were highly accurate for approximating the mutual information between the stimuli and the responses of a large neural population. These approximation formulas may potentially bring convenience to the applications of information theory to many practical and theoretical problems.
登录
查看更多内容
影响因子：
2.4
作者：
Sompolinsky, H;Yoon, H;Shamir, M
通讯作者：
Shamir, M
影响因子：
2.5
作者：
Rissanen, JJ
通讯作者：
Rissanen, JJ
影响因子：
1.8
作者：
Bell, AJ;Sejnowski, TJ
通讯作者：
Sejnowski, TJ
影响因子：
2.5
作者：
CLARKE, BS;BARRON, AR
通讯作者：
BARRON, AR
影响因子：
4.5
作者：
BERAN, R
通讯作者：
BERAN, R