Estimating the Unique Information of Continuous Variables
Estimating the Unique Information of Continuous Variables
复制标题
DOI：
--
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Ari Pakman;D. Gilboa;E. Schneidman
中科院分区：
文献类型：
--
作者：
Ari Pakman;D. Gilboa;E. Schneidman
从多个源到多个目标的信息的整合和传递是神经系统的核心动机。部分信息分解（PID）的新兴领域提供了一个新的信息理论透镜到这些机制，通过识别协同，冗余和独特的贡献，一个和几个变量之间的互信息。虽然许多作品已经研究了高斯和离散分布的PID方面，一般连续分布的情况下仍然是未知的领域。在这项工作中，我们提出了一种方法来估计的唯一信息在连续分布的情况下，一个与两个变量。我们的方法解决了相关的优化问题，通过结合copula分解和技术开发的优化变分自动编码器的分布空间与固定的二元边际。我们获得了良好的协议与已知的高斯分析结果，并说明了我们的新方法在几个大脑启发的神经模型的力量。我们的方法是能够恢复率神经元的混沌网络的有效连接，并揭示了一个复杂的权衡冗余，协同作用和独特的信息在经常性的网络训练，以解决广义XOR任务。
The integration and transfer of information from multiple sources to multiple targets is a core motive of neural systems. The emerging field of partial information decomposition (PID) provides a novel information-theoretic lens into these mechanisms by identifying synergistic, redundant, and unique contributions to the mutual information between one and several variables. While many works have studied aspects of PID for Gaussian and discrete distributions, the case of general continuous distributions is still uncharted territory. In this work we present a method for estimating the unique information in continuous distributions, for the case of one versus two variables. Our method solves the associated optimization problem over the space of distributions with fixed bivariate marginals by combining copula decompositions and techniques developed to optimize variational autoencoders. We obtain excellent agreement with known analytic results for Gaussians, and illustrate the power of our new approach in several brain-inspired neural models. Our method is capable of recovering the effective connectivity of a chaotic network of rate neurons, and uncovers a complex trade-off between redundancy, synergy and unique information in recurrent networks trained to solve a generalized XOR task.