InTrans: Fast Incremental Transformer for Time Series Data Prediction
InTrans: Fast Incremental Transformer for Time Series Data Prediction
复制标题
InTrans：用于时间序列数据预测的快速增量变压器
DOI：
10.1007/978-3-031-12426-6_4
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Hiroyuki Kitagawa
中科院分区：
文献类型：
--
作者：
Savong Bou;Toshiyuki Amagasa;Hiroyuki Kitagawa
时间序列数据预测在自然灾害预防系统、天气预报系统、交通控制系统等方面有着广泛的应用。时间序列预测已被广泛研究。现有的许多预测模型在预测短序列时间序列时往往表现良好。然而，当处理长时间的数据时，它们的性能会大大降低。最近，在这个方向上进行了更多的专门研究，而Informer是目前最有效的预测模型。Informer的主要缺点是不能增量学习。本文提出了一种增量变压器，称为dintrans，通过减少Informer的训练/预测时间来解决上述瓶颈。与Informer相比，InTrans的时间复杂度为：(1)位置和时间嵌入0 (S) vsO(L),(2)值嵌入0 (S) vsO(L),(3)查询/键/值（Query/Key/ value）的计算0 (S) vs (L)，其中，i为输入的长度；为内核大小；为维度数；和是输入中不重叠部分的长度，通常明显小于l。因此，与最先进的Informer模型相比，InTrans可以大大提高训练和预测速度。大量的实验表明，InTrans在短序列和长序列时间序列预测方面都比Informer快26%左右。
Predicting time-series data is useful in many applications, such as natural disaster prevention system, weather forecast, traffic control system, etc. Time-series forecasting has been extensively studied. Many existing forecasting models tend to perform well when predicting short sequence time-series. However, their performances greatly degrade when dealing with the long one. Recently, more dedicated research has been done for this direction, and Informer is currently the most efficient predicting model. The main drawback of Informer is the inability to incrementally learn. This paper proposes an incremental Transformer, calledInTrans, to address the above bottleneck by reducing the training/predicting time of Informer. The time complexities of InTrans comparing to the Informer are: (1)O(S) vsO(L) for positional and temporal embedding, (2)vsfor value embedding, and (3)vsfor the computation of Query/Key/Value, whereLis the length of the input;kis the kernel size;is the number of dimensions; andSis the length of the non-overlapping part of the input that is usually significantly smaller thanL. Therefore, InTrans could greatly improve both training and predicting speed over the state-of-the-art model, Informer. Extensive experiments have shown that InTrans is about 26% faster than Informer for both short sequence and long sequence time-series prediction.