Dissipation of Information in Channels With Input Constraints
Dissipation of Information in Channels With Input Constraints
复制标题
具有输入约束的通道中的信息耗散
DOI：
10.1109/tit.2015.2482978
复制
发表时间：
2014
影响因子：
2.5
通讯作者：
Yihong Wu
中科院分区：
文献类型：
--
作者：
Yury Polyanskiy;Yihong Wu
信息理论的基本原则之一，数据处理不等式表明，任何通道的输出散度不超过输入散度。对于没有输入约束的通道，对这种收缩量的各种估计是已知的，Dobrushin的总变异系数可能是最著名的。本文研究了具有平均投入成本约束的渠道。研究发现，虽然收缩系数通常等于1（没有收缩），但信息仍然会消散。提出了一种非线性函数，即通道的Dobrushin曲线，来量化耗散量。基于耦合参数，开发了评价加性噪声信道Dobrushin曲线的工具。讨论了随机控制中的一些基本应用、吉布斯测度的唯一性和噪声电路的基本极限。应用表明，在n个功率约束中继和高斯信道链中，端到端互信息和最大平方相关衰减为O(log log n/log n)，这与离散信道链中的指数衰减形成鲜明对比。同样，噪声电路（由有界扇入的门组成）和树上的信息广播（有界度）的行为在信噪比（SNR）中不会经历阈值行为。也就是说，与离散信道的情况不同，无论信噪比如何，误码概率都保持在1/2附近。
One of the basic tenets in information theory, the data processing inequality states that the output divergence does not exceed the input divergence for any channel. For channels without input constraints, various estimates on the amount of such contraction are known, Dobrushin's coefficient for the total variation being perhaps the most well-known. This paper investigates channels with an average input cost constraint. It is found that, while the contraction coefficient typically equals one (no contraction), the information nevertheless dissipates. A certain nonlinear function, the Dobrushin curve of the channel, is proposed to quantify the amount of dissipation. Tools for evaluating the Dobrushin curve of additive-noise channels are developed based on coupling arguments. Some basic applications in stochastic control, uniqueness of Gibbs measures, and fundamental limits of noisy circuits are discussed. As an application, it is shown that, in the chain of n power-constrained relays and Gaussian channels, the end-to-end mutual information and maximal squared correlation decay as O(log log n/log n), which is in stark contrast with the exponential decay in chains of discrete channels. Similarly, the behavior of noisy circuits (composed of gates with bounded fan-in) and broadcasting of information on trees (of bounded degree) does not experience threshold behavior in the signal-to-noise ratio (SNR). Namely, unlike the case of discrete channels, the probability of bit error stays bounded away from 1/2 regardless of the SNR.