ITENE: Intrinsic Transfer Entropy Neural Estimator
ITENE: Intrinsic Transfer Entropy Neural Estimator
复制标题
ITENE：固有传递熵神经估计器
DOI：
10.3929/ethz-b-000402960
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
M. Richardson
中科院分区：
文献类型：
--
作者：
Jingjing Zhang;O. Simeone;Z. Cvetković;E. Abela;M. Richardson
量化信息流的方向性有助于理解并可能控制许多复杂系统的运行，如交通、社会、神经或基因调控网络。标准的转移熵（TE）度量遵循格兰杰的因果关系原理，通过测量源信号$X$的过去状态和目标信号$Y$的未来状态之间的互信息（MI），同时以$Y$的过去状态为条件。因此，当除了$Y$的过去之外，还具有来自$X$的可用的过去样本时，TE量化了可以累积的靶序列$Y$的预测中的改善，如通过对数损失测量的。然而，通过以$Y$的过去为条件，TE还测量可以通过观察$X$和$Y$的过去而不仅仅是$X$的过去来协同提取的信息。建立在一个私钥协议的制定，内在TE（ITE）的目的是折扣这样的协同信息，以量化的程度，其中$X$是\单独}预测的$Y$，独立于$Y$的过去。在本文中，ITE的估计提出了最近提出的互信息神经估计（MINE）的启发。该估计是基于KL散度的变分界，双样本神经网络分类器，和蒙特卡罗梯度的路径估计。
Quantifying the directionality of information flow is instrumental in understanding, and possibly controlling, the operation of many complex systems, such as transportation, social, neural, or gene-regulatory networks. The standard Transfer Entropy (TE) metric follows Granger's causality principle by measuring the Mutual Information (MI) between the past states of a source signal $X$ and the future state of a target signal $Y$ while conditioning on past states of $Y$. Hence, the TE quantifies the improvement, as measured by the log-loss, in the prediction of the target sequence $Y$ that can be accrued when, in addition to the past of $Y$, one also has available past samples from $X$. However, by conditioning on the past of $Y$, the TE also measures information that can be synergistically extracted by observing both the past of $X$ and $Y$, and not solely the past of $X$. Building on a private key agreement formulation, the Intrinsic TE (ITE) aims to discount such synergistic information to quantify the degree to which $X$ is \emph{individually} predictive of $Y$, independent of $Y$'s past. In this paper, an estimator of the ITE is proposed that is inspired by the recently proposed Mutual Information Neural Estimation (MINE). The estimator is based on variational bound on the KL divergence, two-sample neural network classifiers, and the pathwise estimator of Monte Carlo gradients.
影响因子：
4.6
作者：
Hawlader, MNA;Uddin, MS;Zhu, HJ
通讯作者：
Zhu, HJ