Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation
Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation
复制标题
DOI：
10.1109/tit.2016.2646342
复制
发表时间：
2017-03-01
影响因子：
2.5
通讯作者：
Raginsky, Maxim
中科院分区：
文献类型：
--
作者：
Xu, Aolin;Raginsky, Maxim
我们得到了分散估计中贝叶斯风险的下界，其中估计量不能直接访问有条件地产生于感兴趣的随机参数上的随机样本，而只能访问从观察样本的局部处理机接收的数据。由于从处理器到估计器的量化和噪声通信信道，所接收的数据受到通信限制。我们首先利用信息论中的互信息、信息密度、小球概率和微分熵等信息量，推导出贝叶斯风险的一般下界。然后，我们将这些下界应用于分散的情况，使用强大的数据处理不等式来量化由于通信限制而导致的信息收缩。对于非交互和交互通信协议，我们处理单个处理机和多个处理机的情况，其中不同处理机观察到的样本可能在给定参数的情况下是有条件地依赖的。我们的结果恢复和改进了最近关于某些分散估计问题的Bayes风险和极小极大风险的下界，其中以前只考虑了条件独立的样本集和无噪声信道。此外，我们的结果提供了一种量化将资源分配到多个处理器导致的估计性能下降的一般方法，这在已有的工作中只针对具体的例子进行了讨论。
We derive lower bounds on the Bayes risk in decentralized estimation, where the estimator does not have direct access to the random samples generated conditionally on the random parameter of interest, but only to the data received from local processors that observe the samples. The received data are subject to communication constraints due to the quantization and the noisy communication channels from the processors to the estimator. We first derive general lower bounds on the Bayes risk using information-theoretic quantities, such as mutual information, information density, small ball probability, and differential entropy. We then apply these lower bounds to the decentralized case, using strong data processing inequalities to quantify the contraction of information due to communication constraints. We treat the cases of a single processor and of multiple processors, where the samples observed by different processors may be conditionally dependent given the parameter, for noninteractive and interactive communication protocols. Our results recover and improve recent lower bounds on the Bayes risk and the minimax risk for certain decentralized estimation problems, where previously only conditionally independent sample sets and noiseless channels have been considered. Moreover, our results provide a general way to quantify the degradation of estimation performance caused by distributing resources to multiple processors, which is only discussed for specific examples in existing works.