On the Rényi Divergence, Joint Range of Relative Entropies, and a Channel Coding Theorem
On the Rényi Divergence, Joint Range of Relative Entropies, and a Channel Coding Theorem
复制标题
关于 Rényi 散度、相对熵的联合范围和通道编码定理
DOI：
--
复制
发表时间：
2015
影响因子：
2.5
通讯作者：
I. Sason
中科院分区：
文献类型：
--
作者：
I. Sason
本文首先考虑在总变差距离的约束下最小化雷尼发散。基于该优化问题的解，当P1、P2和Q是相互绝对连续的任意概率测度，并且P1和P2之间的总变化距离不小于给定值时，确定点（D（QIIP 1），D（QIIP 2））的精确轨迹。它进一步表明，这个凸区域的所有点是由概率措施，这是定义在一个二进制字母表。这种表征产生了一个几何的解释的最小的dupff信息受到约束的变化距离。本文还导出了二元线性分组码（或码集成）在最大似然译码下性能的指数上界。它的推导依赖于Gallager边界技术，它再现了Shulman-Feder边界作为一个特例。该界限用从码的归一化距离谱（或系综的平均距离谱）到随机分组码的容量实现系综的二项分布距离谱的雷尼发散来表示。该指数界限提供了二进制线性分组码（或码集合）的性能退化的定量测量，作为它们的距离谱与二项分布的偏差的函数。被认为是一个有效的使用这个界限。
This paper starts by considering the minimization of the Rényi divergence subject to a constraint on the total variation distance. Based on the solution of this optimization problem, the exact locus of the points (D(QIIP1), D(QIIP2)) is determined when P1, P2, and Q are arbitrary probability measures which are mutually absolutely continuous, and the total variation distance between P1 and P2 is not below a given value. It is further shown that all the points of this convex region are attained by probability measures which are defined on a binary alphabet. This characterization yields a geometric interpretation of the minimal Chernoff information subject to a constraint on the variational distance. This paper also derives an exponential upper bound on the performance of binary linear block codes (or code ensembles) under maximum-likelihood decoding. Its derivation relies on the Gallager bounding technique, and it reproduces the Shulman-Feder bound as a special case. The bound is expressed in terms of the Rényi divergence from the normalized distance spectrum of the code (or the average distance spectrum of the ensemble) to the binomially distributed distance spectrum of the capacity-achieving ensemble of random block codes. This exponential bound provides a quantitative measure of the degradation in performance of binary linear block codes (or code ensembles) as a function of the deviation of their distance spectra from the binomial distribution. An efficient use of this bound is considered.