Renewable estimation and incremental inference in generalized linear models with streaming data sets
Renewable estimation and incremental inference in generalized linear models with streaming data sets
复制标题
DOI：
10.1111/rssb.12352
复制
发表时间：
2019-12-23
影响因子：
5.8
通讯作者：
Song, Peter X-K
中科院分区：
文献类型：
--
作者：
Luo, Lan;Song, Peter X-K
本文提出了一种基于广义线性模型的数据流增量更新算法。该方法是在一个新的可更新估计和增量推理框架下提出的，其中最大似然估计是用当前数据和历史数据的汇总统计量来更新的。我们的框架可以在一个流行的分布式计算环境中实现，称为Apache Spark，以扩展计算。rho体系结构由两个数据处理层组成，它使我们能够容纳与推理相关的统计数据，并便于对估计和推理中使用的统计数据进行顺序更新。我们证明了所提出的可更新估计量的估计相合性和渐近正态性，其中Wald检验用于增量推断.我们的方法进行了检查，并说明了各种数值模拟实验和真实的世界的数据分析的例子。
The paper presents an incremental updating algorithm to analyse streaming data sets using generalized linear models. The method proposed is formulated within a new framework of renewable estimation and incremental inference, in which the maximum likelihood estimator is renewed with current data and summary statistics of historical data. Our framework can be implemented within a popular distributed computing environment, known as Apache Spark, to scale up computation. Consisting of two data-processing layers, the rho architecture enables us to accommodate inference-related statistics and to facilitate sequential updating of the statistics used in both estimation and inference. We establish estimation consistency and asymptotic normality of the proposed renewable estimator, in which the Wald test is utilized for an incremental inference. Our methods are examined and illustrated by various numerical examples from both simulation experiments and a real world data analysis.