CIF: Small: Non-Asymptotic Information Theory

在实时语音和高速数据应用中，有限的延迟是一个关键的设计约束;事实上，在无线系统中，短至几百位的数据包大小是很常见的。本研究的目的是超越传统的改进的基本渐近信息理论的限制和调查的回退容量（在信道编码）和开销超过熵（无损压缩）和率失真函数（在有损信源编码）所引起的编码在一个给定的块长度。我们计划重新审视主要的设计原则，从容量，率失真函数和最小信源编码率的分析，看看他们仍然适用于非渐近制度，对于那些不，评估遵守他们短块长度所产生的惩罚。我们对最佳速率的非渐近行为的研究涉及两个互补的目标：a）可计算的上界和下界足够紧密，以将非渐近操作基本极限的不确定性降低到与长块长渐近性的差距相比可以忽略的水平; B）甚至对于短嵌段长度也是准确的边界的分析近似，以便提供对良好编码策略的洞察并实现实际相关的优化问题。这些近似值通常涉及一个我们称为色散的参数，它量化了信源和信道的随机变化。

In real-time voice and high-speed data applications, limited delay is a key design constraint; indeed, packet sizes as short as a few hundred bits are common in wireless systems. The objective of this research is to go beyond traditional refinements to the fundamental asymptotic information theoretic limits and investigate the back-off from capacity (in channel coding) and the overhead over entropy (in lossless compression) and the rate-distortion function (in lossy source coding) incurred by coding at a given blocklength. We plan to revisit the major design principles stemming from the analysis of capacity, rate-distortion function and minimum source coding rate and see which of them still apply in the non-asymptotic regime, and for those that do not, assess the penalty incurred by abiding by them for short blocklengths. Our study of the non-asymptotic behavior of the optimum rate achievable as a function of both blocklength and error probability involves two complementary goals:a) computable upper and lower bounds tight enough to reduce the uncertainty on the non-asymptotic operational fundamental limit to a level that is negligible compared to the gap to the long-blocklength asymptotics; b) analytical approximations to the bounds that are accurate even for short blocklengths, so as to offer insights into good coding strategies and enable practically relevant optimization problems. Those approximations typically involve a parameter we refer to as dispersion, which quantifies the stochastic variability of sources and channels.