Estimation of non-stationary Markov Chain transition models
Estimation of non-stationary Markov Chain transition models
复制标题
非平稳马尔可夫链转移模型的估计
DOI：
10.1109/cdc.2008.4738904
复制
发表时间：
2008
期刊：
影响因子：
--
通讯作者：
J. How
中科院分区：
文献类型：
--
作者：
L. Bertuccelli;J. How
许多决策系统依赖于精确已知的马尔可夫链模型来保证最优性能，本文考虑了具有完美状态观测的未知非平稳马尔可夫链转移模型的在线估计问题。在使用不确定行上的先验Dirichlet分布时，我们推导出最大后验概率(MAP)估计的均值-方差等价。这种递归均值-方差估计器扩展了以前的方法，这些方法使用观察到的转换计数重新计算每个时间步长的矩。结果表明，这种均值-方差估计器对过渡模型(特别是切换模型)的变化反应较慢，并利用经典滤波中的伪噪声相加思想进行了改进，以加快估计器的响应速度。这种新的折扣均值-方差估计器直观地解释了先前的衰落观测，并提供了与隐马尔可夫模型估计中使用的衰落技术的链接。我们的新估计技术比其他估计技术，如有限记忆估计器，速度更快，误差也更小。
Many decision systems rely on a precisely known Markov Chain model to guarantee optimal performance, and this paper considers the online estimation of unknown, non-stationary Markov Chain transition models with perfect state observation. In using a prior Dirichlet distribution on the uncertain rows, we derive a mean-variance equivalent of the maximum a posteriori (MAP) estimator. This recursive mean-variance estimator extends previous methods that recompute the moments at each time step using observed transition counts. It is shown that this mean-variance estimator responds slowly to changes in transition models (especially switching models) and a modification that uses ideas of pseudonoise addition from classical filtering is used to speed up the response of the estimator. This new, discounted mean-variance estimator has the intuitive interpretation of fading previous observations and provides a link to fading techniques used in Hidden Markov Model estimation. Our new estimation techniques is both faster and has reduced error than alternative estimation techniques, such as finite memory estimators.