Inequalities between entropy and index of coincidence derived from information diagrams
Inequalities between entropy and index of coincidence derived from information diagrams
复制标题
DOI：
10.1109/18.959272
复制
发表时间：
2001-11-01
影响因子：
2.5
通讯作者：
Topsoe, F
中科院分区：
文献类型：
--
作者：
Harremoës, P;Topsoe, F
对于任何离散概率分布 P，我们可以将其熵 H(P) = - Sigma p(i) ln p(i) 及其重合指数 IC(P) = Sigma p(i)(2) 关联起来。论文的主要成果是确定了地图P曲线右箭头（IC(P),H(P)）的精确范围。该范围看起来很像地图 P 弯曲的右箭头（P-max，H（P）），其中 P，R 是最大点概率，参见。从 1965 年（Kovalevskij [18]）到 1994 年（Feder 和 Merhav [7]）的研究。早期的结果实际上关注的是误差概率 1 - P-max 而不是 P-max，可以将其视为通过此处介绍的方法获得的结果的限制情况。所指示的地图范围称为信息图。主要结果给出了熵函数的精确下限和上限。其中一些界限对于伯努利源的通用编码和预测的某些问题的精确解决至关重要。其他应用涉及香农理论（各种散度度量之间的关系）、统计决策理论和率失真理论。开发了两种方法。一是拓扑学的；另一种涉及凸分析，基于“替换引理”，该引理与混合类型优化问题（凹/凸优化）具有独立的兴趣。
To any discrete probability distribution P we can associate its entropy H(P) = - Sigma p(i) ln p(i) and its index of coincidence IC(P) = Sigma p(i)(2). The main result of the paper is the determination of the precise range of the map P curved right arrow (IC(P), H(P)). The range looks much like that of the map P curved right arrow (P-max, H(P)) where P,R is the maximal point probability, cf. research from 1965 (Kovalevskij [18]) to 1994 (Feder and Merhav [7]). The earlier results, which actually focus on the probability of error 1 - P-max rather than P-max can be conceived as limiting cases of results obtained by methods presented here. Ranges of maps as those indicated are called Information Diagrams.The main result gives rise to precise lower as well as upper bounds for the entropy function. Some of these bounds are essential for the exact solution of certain problems of universal coding and prediction for Bernoulli sources. Other applications concern Shannon theory (relations between various measures of divergence), statistical decision theory, and rate distortion theory.Two methods are developed. One is topological; the other involves convex analysis and is based on a "lemma of replacement" which is of independent interest in relation to problems of optimization of mixed type (concave/convex optimization).