Past-future information bottleneck in dynamical systems
Past-future information bottleneck in dynamical systems
复制标题
DOI：
10.1103/physreve.79.041925
复制
发表时间：
2009-04-01
影响因子：
2.4
通讯作者：
Tishby, Naftali
中科院分区：
文献类型：
--
作者：
Creutzig, Felix;Globerson, Amir;Tishby, Naftali
生物系统需要实时处理信息，并且必须权衡表示和编码成本的准确性。在这里，我们将这种权衡付诸实施，并开发一个信息论框架，该框架选择性地提取关于输出未来的输入过去的信息，从而获得一个广义特征值问题。因此，我们根据与状态空间的附加维度相对应的结构相变来解开输入历史。我们阐明了它与典型相关分析的关系，并给出了一个数值例子。总之，这项工作将信息论优化与系统辨识和模型降阶的联合问题联系起来。
Biological systems need to process information in real time and must trade off accuracy of presentation and coding costs. Here we operationalize this trade-off and develop an information-theoretic framework that selectively extracts information of the input past that is predictive about the output future, obtaining a generalized eigenvalue problem. Thereby, we unravel the input history in terms of structural phase transitions corresponding to additional dimensions of a state space. We elucidate the relation to canonical correlation analysis and give a numerical example. Altogether, this work relates information-theoretic optimization to the joint problem of system identification and model reduction.