Information Theoretic Proofs of Entropy Power Inequalities
Information Theoretic Proofs of Entropy Power Inequalities
复制标题
DOI：
10.1109/tit.2010.2090193
复制
发表时间：
2011-01-01
影响因子：
2.5
通讯作者：
Rioul, Olivier
中科院分区：
文献类型：
--
作者：
Rioul, Olivier
虽然大多数有用的信息论不等式可以从熵或互信息的基本性质推导出来，但到目前为止，香农熵幂不等式（EPI）是一个例外：EPI的现有信息论证明依赖于使用Fisher信息或最小均方误差（MMSE）的微分熵表示，这些信息论证明来自de Bruijn恒等式。在本文中，我们首先提出了一个统一的观点，这些证明，表明他们共享两个基本成分：1）应用于协方差保持线性变换的数据处理参数; 2）连续高斯扰动路径上的积分。使用这些成分，我们开发了一个新的和简短的证明EPI通过互信息不等式，取代斯塔姆和布莱克曼的费舍尔信息不等式（FIS）和不等式MMSE郭，沙迈，和Verdu在早期的证明。结果的优点是非常简单，因为它只依赖于互信息的基本属性。这些想法，然后推广到各种扩展版本的EPI：Zamir和Feder的广义EPI的线性变换的随机变量，高野和约翰逊的EPI的因变量，刘和Viswanath的协方差约束EPI，和科斯塔的熵功率的ESTA不等式。
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the EPI hinge on representations of differential entropy using either Fisher information or minimum mean-square error (MMSE), which are derived from de Bruijn's identity. In this paper, we first present an unified view of these proofs, showing that they share two essential ingredients: 1) a data processing argument applied to a covariance-preserving linear transformation; 2) an integration over a path of a continuous Gaussian perturbation. Using these ingredients, we develop a new and brief proof of the EPI through a mutual information inequality, which replaces Stam and Blachman's Fisher information inequality (FII) and an inequality for MMSE by Guo, Shamai, and Verdu used in earlier proofs. The result has the advantage of being very simple in that it relies only on the basic properties of mutual information. These ideas are then generalized to various extended versions of the EPI: Zamir and Feder's generalized EPI for linear transformations of the random variables, Takano and Johnson's EPI for dependent variables, Liu and Viswanath's covariance-constrained EPI, and Costa's concavity inequality for the entropy power.