CIF: Small: Distributed Statistical Inference with Compressed Data

由于数据集的大小和规模的快速增长以及利用多个机器的并行处理能力的期望，分布式统计推断和机器学习（其中可用数据存储在允许以有限的通信预算彼此通信的多个机器中）已经吸引了显著的研究兴趣。分布式设置有两种基本场景：样本分区和特征分区。虽然最近有许多工作的设计推理算法的样本分区的情况下，一直是有限的工作功能分区的情况下。本项目的重点是从信息论的角度来描述特征划分场景的基本限制并开发分布式统计算法。本研究通过关注两个研究重点来解决这些挑战。推力1专注于设计交互式推理编码方案。其主要思想是，通过相互交互，终端可以协调其压缩，使决策者可以获得更多的信息，同时使用相同的通信资源，这将导致更好的推理性能的参数。Thrust 2设计了用于推理的函数计算方案，其中机器计算观测值的函数而不首先恢复它们，然后从该函数执行推理。这个想法的主要动机是恢复观察或它们的压缩版本在分布式推理设置中是不必要的，因为分布式推理的最终目标是推断未知参数的值。

Due to the rapid growth of size and scale of datasets and desire to harnessing parallel processing capabilities of multiple machines, distributed statistical inference and machine learning, in which available data are stored in multiple machines who are allowed to communicate with each other with limited communication budgets, have attracted significant research interests. There are two basic scenarios for the distributed setting: sample partition and feature partition. Although there have been many recent work on the design of inference algorithms for the sample partition scenario, there has been limited work on the feature partition scenario. The focus of this project is to characterize the fundamental limits and develop distributed statistical algorithms for the feature partition scenario from information theoretic perspective.Compared with the sample partition scenario, the feature partition scenario is significantly more challenging. This research addresses these challenges by focusing on two research thrusts. Thrust 1 focuses on designing interactive encoding schemes for inference. The main idea is that, by interacting with each other, the terminals can coordinate their compression so that the decision maker can obtain more information about the parameter while using the same communication resources, which will lead to a better inference performance. Thrust 2 designs function computing schemes for inference, in which the machines compute a function of observations without recovering them first and then perform inference from this function. The main motivation for this idea is that recovering observations or a compressed version of them is not necessary in the distributed inference setup, as the final goal of the distributed inference is to infer the value of the unknown parameter.