A Split-and-Conquer Approach for Analysis of Extraordinarily Large Data
A Split-and-Conquer Approach for Analysis of Extraordinarily Large Data
复制标题
DOI：
10.14288/1.0043878
复制
发表时间：
2014-02
影响因子：
1.4
通讯作者：
Xueying Chen;Min‐ge Xie
中科院分区：
文献类型：
--
作者：
Xueying Chen;Min‐ge Xie
如果有数据集，太大而无法放入一台计算机，或者对于计算密集型数据分析来说太昂贵，我们应该怎么办？我们提出了一个分裂和征服的方法，并说明它使用几个计算密集的惩罚回归方法，沿着的理论支持。我们表明，分裂和征服的方法可以大大减少计算时间和计算机内存的要求。所提出的方法进行了说明，数值模拟和数据的例子。
If there are datasets, too large to fit into a single computer or too expen- sive for a computationally intensive data analysis, what should we do? We propose a split-and-conquer approach and illustrate it using several computationally inten- sive penalized regression methods, along with a theoretical support. We show that the split-and-conquer approach can substantially reduce computing time and com- puter memory requirements. The proposed methodology is illustrated numerically using both simulation and data examples.