CIF: Small: Distributed Statistical Inference with Compressed Data

由于数据集的大小和规模的快速增长以及利用多台机器并行处理能力的愿望，分布式统计推理和机器学习，其中可用数据存储在允许在有限的通信预算下相互通信的多台机器中，已经吸引了重要的研究兴趣。分布式设置有两种基本场景：样本分区和特征分区。尽管最近有许多针对样本分区场景的推理算法设计的工作，但针对特征分区场景的工作却很有限。本项目的重点是从信息论的角度描述特征划分场景的基本限制和开发分布式统计算法。与样例分区场景相比，特征分区场景明显更具挑战性。本研究通过关注两个研究重点来解决这些挑战。推力1侧重于设计用于推理的交互式编码方案。其主要思想是，通过终端之间的相互作用，协调压缩，使决策者在使用相同的通信资源的情况下获得更多的参数信息，从而获得更好的推理性能。Thrust 2设计了用于推理的函数计算方案，在该方案中，机器先计算观测值的函数，而不首先恢复它们，然后从该函数进行推理。这个想法的主要动机是，在分布式推理设置中，恢复观测值或它们的压缩版本是不必要的，因为分布式推理的最终目标是推断未知参数的值。

Due to the rapid growth of size and scale of datasets and desire to harnessing parallel processing capabilities of multiple machines, distributed statistical inference and machine learning, in which available data are stored in multiple machines who are allowed to communicate with each other with limited communication budgets, have attracted significant research interests. There are two basic scenarios for the distributed setting: sample partition and feature partition. Although there have been many recent work on the design of inference algorithms for the sample partition scenario, there has been limited work on the feature partition scenario. The focus of this project is to characterize the fundamental limits and develop distributed statistical algorithms for the feature partition scenario from information theoretic perspective.Compared with the sample partition scenario, the feature partition scenario is significantly more challenging. This research addresses these challenges by focusing on two research thrusts. Thrust 1 focuses on designing interactive encoding schemes for inference. The main idea is that, by interacting with each other, the terminals can coordinate their compression so that the decision maker can obtain more information about the parameter while using the same communication resources, which will lead to a better inference performance. Thrust 2 designs function computing schemes for inference, in which the machines compute a function of observations without recovering them first and then perform inference from this function. The main motivation for this idea is that recovering observations or a compressed version of them is not necessary in the distributed inference setup, as the final goal of the distributed inference is to infer the value of the unknown parameter.