CIF: Small: Distributed Statistical Inference with Compressed Data

CIF：小型：使用压缩数据进行分布式统计推断

基本信息

批准号：
1717943
负责人：
Lifeng Lai
金额：
$ 45万
依托单位：
University of California-Davis
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-07-15 至 2022-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717943&HistoricalAwards=false
关键词：
CIF Small Distributed Statistical Inference

项目摘要

Due to the rapid growth of size and scale of datasets and desire to harnessing parallel processing capabilities of multiple machines, distributed statistical inference and machine learning, in which available data are stored in multiple machines who are allowed to communicate with each other with limited communication budgets, have attracted significant research interests. There are two basic scenarios for the distributed setting: sample partition and feature partition. Although there have been many recent work on the design of inference algorithms for the sample partition scenario, there has been limited work on the feature partition scenario. The focus of this project is to characterize the fundamental limits and develop distributed statistical algorithms for the feature partition scenario from information theoretic perspective.Compared with the sample partition scenario, the feature partition scenario is significantly more challenging. This research addresses these challenges by focusing on two research thrusts. Thrust 1 focuses on designing interactive encoding schemes for inference. The main idea is that, by interacting with each other, the terminals can coordinate their compression so that the decision maker can obtain more information about the parameter while using the same communication resources, which will lead to a better inference performance. Thrust 2 designs function computing schemes for inference, in which the machines compute a function of observations without recovering them first and then perform inference from this function. The main motivation for this idea is that recovering observations or a compressed version of them is not necessary in the distributed inference setup, as the final goal of the distributed inference is to infer the value of the unknown parameter.

由于数据集的大小和规模的快速增长，并渴望利用多台机器的并行处理能力，分布式统计推断和机器学习，其中可用的数据存储在多个机器中，这些机器被允许与沟通预算有限的相互通信相互通信，吸引了大量的研究兴趣。分布式设置有两个基本方案：示例分区和特征分区。尽管最近在示例分区方案的推理算法设计上进行了许多工作，但在功能分区方案上的工作有限。该项目的重点是表征基本限制，并从信息理论的角度来为特征分区方案开发分布式统计算法，并具有示例分区方案的比例，功能分区方案明显更具挑战性。这项研究通过关注两个研究推力来解决这些挑战。推力1专注于设计用于推理的交互式编码方案。主要思想是，通过相互交互，终端可以协调其压缩，以便决策者在使用相同的通信资源的同时获得有关参数的更多信息，这将带来更好的推理性能。推力2设计功能计算推理的功能，其中计算机计算观测值的函数而无需先恢复它们，然后从此函数中执行推理。这个想法的主要动机是，在分布式推理设置中恢复观测值或压缩版本是不需要的，因为分布式推理的最终目标是推断未知参数的值。