STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction-猫眼课题宝

DOI：

10.48550/arxiv.2312.17346

发表时间：

2023-12

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Dennis Wu;Jerry Yao-Chieh Hu;Weijian Li;Bo-Yu Chen;Han Liu

我们提出了 STanHop-Net（稀疏串联 Hopfield 网络），用于具有内存增强功能的多元时间序列预测。我们方法的核心是 STanHop，一种新颖的基于 Hopfield 的神经网络块，它以数据依赖的方式稀疏地学习和存储时间和跨序列表示。本质上，STanHop 使用两个串联稀疏 Hopfield 层顺序学习时间表示和跨系列表示。此外，StanHop 还包含两个额外的外部内存模块：即插即用模块和调谐即用模块，分别用于无训练和任务感知内存增强。它们使 StanHop-Net 能够快速响应某些突发事件。在方法上，我们通过以分层方式堆叠 STanHop 块来构建 StanHop-Net，从而实现具有特定于分辨率的稀疏性的多分辨率特征提取。理论上，我们引入了现代 Hopfield 模型的稀疏扩展（广义稀疏现代 Hopfield 模型），并表明与密集模型相比，它在不牺牲记忆容量的情况下赋予了更严格的记忆检索错误。根据经验，我们验证了我们的框架在合成和现实环境中的有效性。

We present STanHop-Net (Sparse Tandem Hopfield Network) for multivariate time series prediction with memory-enhanced capabilities. At the heart of our approach is STanHop, a novel Hopfield-based neural network block, which sparsely learns and stores both temporal and cross-series representations in a data-dependent fashion. In essence, STanHop sequentially learn temporal representation and cross-series representation using two tandem sparse Hopfield layers. In addition, StanHop incorporates two additional external memory modules: a Plug-and-Play module and a Tune-and-Play module for train-less and task-aware memory-enhancements, respectively. They allow StanHop-Net to swiftly respond to certain sudden events. Methodologically, we construct the StanHop-Net by stacking STanHop blocks in a hierarchical fashion, enabling multi-resolution feature extraction with resolution-specific sparsity. Theoretically, we introduce a sparse extension of the modern Hopfield model (Generalized Sparse Modern Hopfield Model) and show that it endows a tighter memory retrieval error compared to the dense counterpart without sacrificing memory capacity. Empirically, we validate the efficacy of our framework on both synthetic and real-world settings.

STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction

文献摘要

相似文献