Long-term prediction of μECOG signals with a spatio-temporal pyramid of adversarial convolutional networks
Long-term prediction of μECOG signals with a spatio-temporal pyramid of adversarial convolutional networks
复制标题
使用对抗性卷积网络的时空金字塔对 μECOG 信号进行长期预测
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
J. Viventi
中科院分区：
文献类型：
--
作者：
Ran Wang;Yilin Song;Yao Wang;J. Viventi
视频预测在很长一段时间内具有许多潜在的应用前景。使用卷积神经网络结构对时间序列的长期动态建模具有挑战性，卷积神经网络结构通常有利于捕获短期依赖关系。在这项工作中，我们建议将卷积神经网络嵌入时空金字塔结构中，以利用长期和短期的时间依赖性，并捕获宏观尺度和微观尺度的空间结构。给定尺度下的预测取决于从较低尺度提取的特征和当前尺度的过去观测。为了克服均方误差损失造成的模糊问题，我们添加了一个基于Wasserstein距离的对抗损失批评模型来补充MSE。我们将我们的时空金字塔模型与单尺度卷积网络以及仅具有多个空间尺度的模型进行了比较，并证明我们的金字塔结构在预测多达24个未来帧方面表现更好。
Video prediction into sufficiently long future has many potential applications. Modeling long-term dynamics for times series is challenging with convolution neural network structure, which is usually good for capturing short-term dependencies. In this work, we propose to embed the convolutional neural network within a spatial-temporal pyramid structure, to exploit both long-term and short-term temporal dependency and capture both macro-scale and micro-scale spatial structures. The prediction at a given scale is conditioned on the features extracted from a lower scale and past observations from the current scale. In order to overcome the blurry issue caused by the mean square error loss, we add a critic model with Wasserstein distance based adversarial loss to complement MSE. We compare our spatio-temporal pyramid model against a single scale convolution network as well as a model with multiple spatial scales only, and demonstrate that our pyramid structure performs better for predicting up to 24 future frames.