Cogra: Concept-Drift-Aware Stochastic Gradient Descent for Time-Series Forecasting
Cogra: Concept-Drift-Aware Stochastic Gradient Descent for Time-Series Forecasting
复制标题
DOI：
10.1609/aaai.v33i01.33014594
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Kohei Miyaguchi;Hiroshi Kajino
中科院分区：
文献类型：
--
作者：
Kohei Miyaguchi;Hiroshi Kajino
我们通过随机梯度下降（SGD）的自动学习率调整来解决存在概念漂移的时间序列预测问题。基于SGD的方法优于其他概念漂移算法，因为它可以应用于任何模型，并且可以在在线预测的同时保持有效的学习。在众多的SGD算法中，基于方差的SGD（vSGD）可以通过自动学习率调整成功地处理概念漂移，这被简化为一个自适应均值估计问题。然而，它的性能仍然是有限的，因为它的启发式均值估计。在本文中，我们提出了一个概念漂移感知的随机梯度下降（Cogra），配备了更理论上健全的平均估计称为序贯均值跟踪（SMT）。我们的主要贡献是，我们定义了一个良好的标准，平均估计SMT的设计是最佳的，根据这个标准。综合实验的结果，我们发现：（i）在存在概念漂移的情况下，我们的SMT可以比vSGD的估计器更好地估计均值;（ii）在预测性能方面，Cogra将真实世界数据集的预测损失减少了16-67%，表明SMT显著提高了预测精度。
We approach the time-series forecasting problem in the presence of concept drift by automatic learning rate tuning of stochastic gradient descent (SGD). The SGD-based approach is preferable to other concept drift algorithms in that it can be applied to any model and it can keep learning efficiently whilst predicting online. Among a number of SGD algorithms, the variance-based SGD (vSGD) can successfully handle concept drift by automatic learning rate tuning, which is reduced to an adaptive mean estimation problem. However, its performance is still limited because of its heuristic mean estimator. In this paper, we present a concept-drift-aware stochastic gradient descent (Cogra), equipped with more theoretically-sound mean estimator called sequential mean tracker (SMT). Our key contribution is that we define a goodness criterion for the mean estimators; SMT is designed to be optimal according to this criterion. As a result of comprehensive experiments, we find that (i) our SMT can estimate the mean better than vSGD’s estimator in the presence of concept drift, and (ii) in terms of predictive performance, Cogra reduces the predictive loss by 16–67% for real-world datasets, indicating that SMT improves the prediction accuracy significantly.