Learning Distributions from their Samples under Communication Constraints
Learning Distributions from their Samples under Communication Constraints
复制标题
在通信约束下从样本中学习分布
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Ayfer Özgür
中科院分区：
文献类型：
--
作者：
L. P. Barnes;Yanjun Han;Ayfer Özgür
我们考虑在分布式网络中学习高维，非参数和结构化（例如高斯）分布的问题，其中网络中的每个节点观察来自底层分布的独立样本，并且可以使用$k$ bits将其样本传送到中央处理器。我们考虑三种不同的通信模式。在独立模型下，每个节点通过将其样本独立编码为$k$位来将其样本传送到中央处理器。在更一般的顺序或黑板通信模型下，节点可以交互地共享信息，但每个节点被限制在最多写入$k$位的最终成绩单。我们刻画了通信约束k对估计潜在分布的极大极小风险的影响。我们开发的极大极小下界，适用于在一个统一的方式，许多常见的统计模型，并揭示了通信约束的影响可以是定性不同的，这取决于与每个模型相关联的得分函数的尾部行为。在我们的证明中的一个关键因素是量化样本的Fisher信息的几何特征。
We consider the problem of learning high-dimensional, nonparametric and structured (e.g. Gaussian) distributions in distributed networks, where each node in the network observes an independent sample from the underlying distribution and can use $k$ bits to communicate its sample to a central processor. We consider three different models for communication. Under the independent model, each node communicates its sample to a central processor by independently encoding it into $k$ bits. Under the more general sequential or blackboard communication models, nodes can share information interactively but each node is restricted to write at most $k$ bits on the final transcript. We characterize the impact of the communication constraint $k$ on the minimax risk of estimating the underlying distribution under $\ell^2$ loss. We develop minimax lower bounds that apply in a unified way to many common statistical models and reveal that the impact of the communication constraint can be qualitatively different depending on the tail behavior of the score function associated with each model. A key ingredient in our proof is a geometric characterization of Fisher information from quantized samples.
Blackboard 通信协议下分布式估计的 Fisher 信息
DOI：
--
发表时间：
2019
期刊：
Abstracts of papers - IEEE International Symposium on Information Theory
影响因子：
--
作者：
Leighton Pate Barnes, Yanjun Han
通讯作者：
Leighton Pate Barnes, Yanjun Han
影响因子：
2.5
作者：
Jayadev Acharya;C. Canonne;Himanshu Tyagi
通讯作者：
Jayadev Acharya;C. Canonne;Himanshu Tyagi