A SPLIT-AND-CONQUER APPROACH FOR ANALYSIS OF EXTRAORDINARILY LARGE DATA
A SPLIT-AND-CONQUER APPROACH FOR ANALYSIS OF EXTRAORDINARILY LARGE DATA
复制标题
DOI：
10.5705/ss.2013.088
复制
发表时间：
2014-10-01
影响因子：
1.4
通讯作者：
Xie, Min-ge
中科院分区：
文献类型：
--
作者：
Chen, Xueying;Xie, Min-ge
如果数据集太大，无法放入一台计算机，或者对于计算密集型数据分析来说太昂贵，我们应该怎么办？我们提出了一个分裂和征服的方法，并说明它使用几个计算密集型惩罚回归方法，沿着与理论支持。我们表明，分裂和征服的方法可以大大减少计算时间和计算机内存的要求。所提出的方法进行了说明，数值模拟和数据的例子。
If there are datasets, too large to fit into a single computer or too expensive for a computationally intensive data analysis, what should we do? We propose a split-and-conquer approach and illustrate it using several computationally intensive penalized regression methods, along with a theoretical support. We show that the split-and-conquer approach can substantially reduce computing time and computer memory requirements. The proposed methodology is illustrated numerically using both simulation and data examples.