CIF: Small: Distributed Statistical Inference with Compressed Data

由于数据集的规模和规模的快速增长以及利用多台机器的并行处理能力的需求，分布式统计推理和机器学习(将可用数据存储在多台机器上，允许在有限的通信预算下相互通信)引起了人们极大的研究兴趣。分布式设置有两个基本场景：样本分区和功能分区。尽管最近有许多关于样本划分场景的推理算法设计的工作，但关于特征划分场景的工作有限。该项目的重点是从信息论的角度刻画基本极限并为特征划分场景开发分布式统计算法，与样本划分场景相比，特征划分场景具有更大的挑战性。这项研究通过关注两个研究方向来应对这些挑战。推力1专注于设计用于推理的交互编码方案。其主要思想是，通过相互交互，终端可以协调它们的压缩，以便决策者在使用相同的通信资源的同时获得更多关于参数的信息，从而产生更好的推理性能。推力2设计了用于推理的函数计算方案，其中机器计算观测的函数而不首先恢复它们，然后从该函数执行推理。这个想法的主要动机是，在分布式推理设置中，恢复观测或其压缩版本不是必需的，因为分布式推理的最终目标是推断未知参数的值。

Due to the rapid growth of size and scale of datasets and desire to harnessing parallel processing capabilities of multiple machines, distributed statistical inference and machine learning, in which available data are stored in multiple machines who are allowed to communicate with each other with limited communication budgets, have attracted significant research interests. There are two basic scenarios for the distributed setting: sample partition and feature partition. Although there have been many recent work on the design of inference algorithms for the sample partition scenario, there has been limited work on the feature partition scenario. The focus of this project is to characterize the fundamental limits and develop distributed statistical algorithms for the feature partition scenario from information theoretic perspective.Compared with the sample partition scenario, the feature partition scenario is significantly more challenging. This research addresses these challenges by focusing on two research thrusts. Thrust 1 focuses on designing interactive encoding schemes for inference. The main idea is that, by interacting with each other, the terminals can coordinate their compression so that the decision maker can obtain more information about the parameter while using the same communication resources, which will lead to a better inference performance. Thrust 2 designs function computing schemes for inference, in which the machines compute a function of observations without recovering them first and then perform inference from this function. The main motivation for this idea is that recovering observations or a compressed version of them is not necessary in the distributed inference setup, as the final goal of the distributed inference is to infer the value of the unknown parameter.