Strong Data Processing Inequalities for Input Constrained Additive Noise Channels
Strong Data Processing Inequalities for Input Constrained Additive Noise Channels
复制标题
DOI：
10.1109/tit.2017.2782359
复制
发表时间：
2018-03-01
影响因子：
2.5
通讯作者：
Wu, Yihong
中科院分区：
文献类型：
--
作者：
Calmon, Flavio du Pin;Polyanskiy, Yury;Wu, Yihong
本文通过非线性强数据处理不等式，将加噪减少可用信息这一直观现象量化。考虑随机变量W -> X -> Y构成一个马氏链，其中Y = X+Z，X和Z为真实的值，独立且X在L-p-范数下有界.证明了I（W; Y）0，当且仅当Z有一个密度，其支撑与其自身的任何平移都不相交。一个相关的问题是描述什么样的耦合（W，X）的互信息I（W; Y）接近最大可能。为此，我们表明，为了使通道饱和，即，为了使I（W; Y）接近容量，必须使I（W; X）->无穷大（在信道上的适当条件下）。这个结果的一个关键成分是反卷积引理，它表明后卷积总变差距离限制了卷积前Kolmogorov-Smirnov距离。对于加性高斯噪声信道中具有二次代价约束的特殊情况，给出了其上界。这些界限被证明是顺序最优的。对于这种情况下，简化的证明提供利用高斯特定的工具，如信息和估计（I-MMSE）和Talagrand的信息传输不等式之间的连接。
This paper quantifies the intuitive observation that adding noise reduces available information by means of nonlinear strong data processing inequalities. Consider the random variables W -> X -> Y forming a Markov chain, where Y = X+Z with X and Z real valued, independent and X bounded in L-p-norm. It is shown that I (W; Y) 0, if and only if Z has a density whose support is not disjoint from any translate of itself. A related question is to characterize for what couplings (W, X) the mutual information I (W; Y) is close to maximum possible. To that end we show that in order to saturate the channel, i.e., for I (W; Y) to approach capacity, it is mandatory that I (W; X) -> infinity (under suitable conditions on the channel). A key ingredient for this result is a deconvolution lemma which shows that postconvolution total variation distance bounds the preconvolution Kolmogorov-Smirnov distance. Explicit bounds are provided for the special case of the additive Gaussian noise channel with quadratic cost constraint. These bounds are shown to be order optimal. For this case, simplified proofs are provided leveraging Gaussian-specific tools such as the connection between information and estimation (I-MMSE) and Talagrand's information-transportation inequality.