Generalized massive optimal data compression
Generalized massive optimal data compression
复制标题
广义海量最优数据压缩
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
B. Wandelt
中科院分区：
文献类型：
--
作者：
Justin Alsing;B. Wandelt
数据压缩已成为现代天文数据分析的基石之一，绝大多数分析都将大型原始数据集压缩为可管理的信息摘要。在本文中，我们提供了一个一般的程序，最佳压缩$N$数据下降到$N$汇总统计，其中$N$是等于感兴趣的参数的数量。我们表明，压缩的得分函数-梯度的对数似然相对于参数-产生$n$压缩统计是最佳的意义上说，他们保留了Fisher信息内容的数据。我们的方法概括了早期工作的线性Karhunen-Lo '{e}ve压缩高斯数据，同时恢复无损线性压缩和二次估计作为特殊情况下，当他们是最佳的。我们给出了一个统一的处理，也包括一般的非高斯的情况下，只要温和的正则性条件得到满足，适当时产生最佳的非线性汇总统计量。作为一个工作的例子，我们推导出明确的$n$最佳压缩统计的高斯数据的平均值和协方差依赖于参数的一般情况下。
Data compression has become one of the cornerstones of modern astronomical data analysis, with the vast majority of analyses compressing large raw datasets down to a manageable number of informative summaries. In this paper we provide a general procedure for optimally compressing $N$ data down to $n$ summary statistics, where $n$ is equal to the number of parameters of interest. We show that compression to the score function -- the gradient of the log-likelihood with respect to the parameters -- yields $n$ compressed statistics that are optimal in the sense that they preserve the Fisher information content of the data. Our method generalizes earlier work on linear Karhunen-Lo'{e}ve compression for Gaussian data whilst recovering both lossless linear compression and quadratic estimation as special cases when they are optimal. We give a unified treatment that also includes the general non-Gaussian case as long as mild regularity conditions are satisfied, producing optimal non-linear summary statistics when appropriate. As a worked example, we derive explicitly the $n$ optimal compressed statistics for Gaussian data in the general case where both the mean and covariance depend on the parameters.