Information-Theoretic Bounds and Approximations in Neural Population Coding.
Information-Theoretic Bounds and Approximations in Neural Population Coding.
复制标题
DOI：
10.1162/neco_a_01056
复制
发表时间：
2018-04
影响因子：
2.9
通讯作者：
Zhang K
中科院分区：
文献类型：
--
作者：
Huang W;Zhang K
虽然香农互信息在许多学科中有着广泛的应用，但在实际应用中，由于维数灾难，高维变量通常很难准确计算其值。这篇文章主要关注在神经群体编码的背景下评估互信息的有效近似方法。对于大但有限的神经种群，我们推导出几个信息理论的渐近界和近似公式，在高维空间中仍然有效。我们证明，优化人口密度分布的基础上，这些近似公式是一个凸优化问题，允许有效的数值解。数值模拟结果证实，我们的渐近公式是高度准确的近似大神经种群的互信息。在特殊情况下，近似公式恰好等于真实的互信息。我们还讨论了变量变换和降维技术，以方便计算的近似。
While Shannon’s mutual information has widespread applications in many disciplines, for practical applications it is often difficult to calculate its value accurately for high-dimensional variables because of the curse of dimensionality. This article focuses on effective approximation methods for evaluating mutual information in the context of neural population coding. For large but finite neural populations, we derive several information-theoretic asymptotic bounds and approximation formulas that remain valid in high-dimensional spaces. We prove that optimizing the population density distribution based on these approximation formulas is a convex optimization problem that allows efficient numerical solutions. Numerical simulation results confirmed that our asymptotic formulas were highly accurate for approximating mutual information for large neural populations. In special cases, the approximation formulas are exactly equal to the true mutual information. We also discuss techniques of variable transformation and dimensionality reduction to facilitate computation of the approximations.
登录
查看更多内容
影响因子：
2.9
作者：
Ganguli, Deep;Simoncelli, Eero P.
通讯作者：
Simoncelli, Eero P.
影响因子：
1.8
作者：
Bell, AJ;Sejnowski, TJ
通讯作者：
Sejnowski, TJ
影响因子：
2.9
作者：
Atick, Joseph J.;Redlich, A. Norman
通讯作者：
Redlich, A. Norman
影响因子：
2.5
作者：
CLARKE, BS;BARRON, AR
通讯作者：
BARRON, AR
影响因子：
64.8
作者：
BECKER, S;HINTON, GE
通讯作者：
HINTON, GE