Markovian Gaussian Process Variational Autoencoders
Markovian Gaussian Process Variational Autoencoders
复制标题
DOI：
10.48550/arxiv.2207.05543
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Harrison Zhu;Carles Balsells Rodas;Yingzhen Li
中科院分区：
文献类型：
--
作者：
Harrison Zhu;Carles Balsells Rodas;Yingzhen Li
序列VAEs已经成功地用于许多高维时间序列建模问题，其中许多变体模型依赖于离散时间机制，如循环神经网络（rnn）。另一方面，连续时间方法最近获得了吸引力，特别是在不规则采样时间序列的背景下，它们可以比离散时间方法更好地处理数据。其中一类是高斯过程变分自编码器（GPVAEs），其中VAE先验被设置为高斯过程（GP）。然而，GPVAEs的一个主要限制是它继承了gp的立方计算成本，这使得它对实践者没有吸引力。在这项工作中，我们利用等效的马尔可夫GPs离散状态空间表示，通过卡尔曼滤波和平滑实现线性时间GPVAE训练。对于我们的模型，马尔可夫GPVAE (MGPVAE)，我们在各种高维时间和时空任务上展示了我们的方法与现有方法相比表现良好，同时在计算上具有高度可扩展性。
Sequential VAEs have been successfully considered for many high-dimensional time series modelling problems, with many variant models relying on discrete-time mechanisms such as recurrent neural networks (RNNs). On the other hand, continuous-time methods have recently gained attraction, especially in the context of irregularly-sampled time series, where they can better handle the data than discrete-time methods. One such class are Gaussian process variational autoencoders (GPVAEs), where the VAE prior is set as a Gaussian process (GP). However, a major limitation of GPVAEs is that it inherits the cubic computational cost as GPs, making it unattractive to practioners. In this work, we leverage the equivalent discrete state space representation of Markovian GPs to enable linear time GPVAE training via Kalman filtering and smoothing. For our model, Markovian GPVAE (MGPVAE), we show on a variety of high-dimensional temporal and spatiotemporal tasks that our method performs favourably compared to existing approaches whilst being computationally highly scalable.