Variational learning for switching state-space models
Variational learning for switching state-space models
复制标题
DOI：
10.1162/089976600300015619
复制
发表时间：
2000-04-01
影响因子：
2.9
通讯作者：
Hinton, GE
中科院分区：
文献类型：
--
作者：
Ghahramani, Z;Hinton, GE
我们引入了一种新的时间序列统计模型，该模型迭代地将数据分割成具有近似线性动力学的区域，并学习这些线性区域的参数。该模型结合并推广了两种最广泛使用的随机时间序列模型-隐马尔可夫模型和线性动力系统-并且与控制和计量经济学文献中广泛使用的模型密切相关。它也可以通过将混合专家神经网络（Jacobs, Jordan, nolan, & Hinton, 1991）扩展到完全动态的版本来推导，其中专家和门控网络都是循环的。该模型隐藏状态的后验概率难以计算，因此不能应用精确期望最大化（EM）算法。然而，我们提出了一种变分近似，使对数似然的下界最大化，并利用隐马尔可夫模型的前向和后向递归和线性动力系统的卡尔曼滤波递归。我们在人工数据集和睡眠呼吸暂停患者的呼吸力自然数据集上测试了该算法。结果表明，变分逼近是切换状态空间模型推理和学习的一种可行方法。
We introduce a new statistical model for time series that iteratively segments data into regimes with approximately linear dynamics and learns the parameters of each of these linear regimes. This model combines and generalizes two of the most widely used stochastic time-series models-hidden Markov models and linear dynamical systems-and is closely related to models that are widely used in the control and econometrics literatures. It can also be derived by extending the mixture of experts neural network (Jacobs, Jordan, Nowlan, & Hinton, 1991) to its fully dynamical version, in which both expert and gating networks are recurrent. Inferring the posterior probabilities of the hidden states of this model is computationally intractable, and therefore the exact expectation maximization (EM) algorithm cannot be applied. However, we present a variational approximation that maximizes a lower bound on the log-likelihood and makes use of both the forward and backward recursions for hidden Markov models and the Kalman filter recursions for linear dynamical systems. We tested the algorithm on artificial data sets and a natural data set of respiration force from a patient with sleep apnea. The results suggest that variational approximations are a viable method for inference and learning in switching state-space models.