Directed Data-Processing Inequalities for Systems with Feedback.
Directed Data-Processing Inequalities for Systems with Feedback.
复制标题
DOI：
10.3390/e23050533
复制
发表时间：
2021-04-26
期刊：
影响因子：
--
通讯作者：
Østergaard J
中科院分区：
文献类型：
--
作者：
Derpich MS;Østergaard J
我们提出了新的数据处理不等式的互信息和有向信息的反馈系统。这种系统中的内部确定性块仅限于因果映射，但允许是非线性和时变的，并且由它们自己的外部随机输入随机化，可以产生任何随机映射。例如，这些随机化块可以代表源编码器、解码器甚至通信信道。此外，所涉及的信号可以任意分布。我们的第一个主要结果涉及相互和定向信息，可以解释为信息流守恒定律。我们的第二个主要结果是一对数据处理不等式（一个是另一个的条件版本）之间的嵌套对完全在闭环随机序列。我们的第三个主要结果介绍和表征的概念，在环（ITL）传输速率的信道编码的情况下，消息是内部的循环。有趣的是，在这种情况下，传统的概念与熵的消息和信道容量的基础上最大化的消息和输出之间的互信息的传输速率是不够的。相反，如我们所示，ITL传输速率是唯一的速率概念，当且仅当这样的ITL速率不超过从消息到解码消息的相应的定向信息速率时，信道码达到零错误概率。我们应用我们的数据处理不等式表明，可实现的（在通常的信道编码意义上）ITL传输速率的上确界是由整个通信信道的有向信息速率的上确界。此外，我们提出了一个例子，在这个上限达到。最后，我们进一步说明了我们的结果的适用性，讨论他们如何使网络控制文献中已知的两个基本不等式的推广成为可能。
We present novel data-processing inequalities relating the mutual information and the directed information in systems with feedback. The internal deterministic blocks within such systems are restricted only to be causal mappings, but are allowed to be non-linear and time varying, and randomized by their own external random input, can yield any stochastic mapping. These randomized blocks can for example represent source encoders, decoders, or even communication channels. Moreover, the involved signals can be arbitrarily distributed. Our first main result relates mutual and directed information and can be interpreted as a law of conservation of information flow. Our second main result is a pair of data-processing inequalities (one the conditional version of the other) between nested pairs of random sequences entirely within the closed loop. Our third main result introduces and characterizes the notion of in-the-loop (ITL) transmission rate for channel coding scenarios in which the messages are internal to the loop. Interestingly, in this case the conventional notions of transmission rate associated with the entropy of the messages and of channel capacity based on maximizing the mutual information between the messages and the output turn out to be inadequate. Instead, as we show, the ITL transmission rate is the unique notion of rate for which a channel code attains zero error probability if and only if such an ITL rate does not exceed the corresponding directed information rate from messages to decoded messages. We apply our data-processing inequalities to show that the supremum of achievable (in the usual channel coding sense) ITL transmission rates is upper bounded by the supremum of the directed information rate across the communication channel. Moreover, we present an example in which this upper bound is attained. Finally, we further illustrate the applicability of our results by discussing how they make possible the generalization of two fundamental inequalities known in networked control literature.
登录
查看更多内容
影响因子：
6.8
作者：
Martins, Nuno C.;Dahleh, Munther A.
通讯作者：
Dahleh, Munther A.
影响因子：
2.5
作者：
Kim, Young-Han
通讯作者：
Kim, Young-Han
影响因子：
1.2
作者：
Quinn CJ;Coleman TP;Kiyavash N;Hatsopoulos NG
通讯作者：
Hatsopoulos NG
DOI：
10.1109/tpami.2021.3063604
发表时间：
2022-03-03
影响因子：
23.6
作者：
Li, Chongyi;Guo, Chunle;Loy, Chen Change
通讯作者：
Loy, Chen Change
影响因子：
2.5
作者：
Kostina, Victoria;Verdu, Sergio
通讯作者：
Verdu, Sergio