Data Processing Theorems and the Second Law of Thermodynamics
Data Processing Theorems and the Second Law of Thermodynamics
复制标题
数据处理定理和热力学第二定律
DOI：
10.1109/tit.2011.2159052
复制
发表时间：
2010
影响因子：
2.5
通讯作者：
N. Merhav
中科院分区：
文献类型：
--
作者：
N. Merhav
我们绘制了Zakai和Ziv（1973年和1975年）的广义数据处理定理与热力学第二定律的动态版本，即玻尔兹曼H定理之间的关系，该定理断言，属于有限状态马尔可夫过程{Xt}的香农熵H（Xt）作为时间t的函数是单调非递减的。前提是该过程的稳态分布在整个状态空间中是均匀的（当过程指定一个孤立系统时就是这种情况）。结果表明，广义数据处理定理和玻尔兹曼h定理都可以看作是应用于马尔可夫过程的某一广义信息测度的单调性（在时间上）的更一般原理的特例。这引起了对广义数据处理定理的新看法，该定理建议对定义广义互信息的凸函数的给定选择利用可能导致更好边界的某些自由度。实际上，我们展示了一个联合源信道编码的特定设置的例子，其中这个想法产生了一个改进的失真下界，相对于1973年的Ziv-Zakai下界和从普通数据处理定理得到的下界。
We draw relationships between the generalized data processing theorems of Zakai and Ziv (1973 and 1975) and the dynamical version of the second law of thermodynamics, a.k.a. the Boltzmann H-Theorem, which asserts that the Shannon entropy, H(Xt), pertaining to a finite-state Markov process {Xt}, is monotonically nondecreasing as a function of time t, provided that the steady-state distribution of this process is uniform across the state space (which is the case when the process designates an isolated system). It turns out that both the generalized data processing theorems and the Boltzmann H-Theorem can be viewed as special cases of a more general principle concerning the monotonicity (in time) of a certain generalized information measure applied to a Markov process. This gives rise to a new look at the generalized data processing theorem, which suggests to exploit certain degrees of freedom that may lead to better bounds, for a given choice of the convex function that defines the generalized mutual information. Indeed, we demonstrate an example of a certain setup of joint source-channel coding, where this idea yields an improved lower bound on the distortion, relative to both the 1973 Ziv-Zakai lower bound and the lower bound obtained from the ordinary data processing theorem.