Analysis of an identification algorithm arising in the adaptive estimation of Markov chains
Analysis of an identification algorithm arising in the adaptive estimation of Markov chains
复制标题
马尔可夫链自适应估计中的辨识算法分析
DOI：
--
复制
发表时间：
1985
期刊：
影响因子：
--
通讯作者：
S. Marcus
中科院分区：
文献类型：
--
作者：
A. Arapostathis;S. Marcus
我们研究了一种应用于部分观察的有限状态马尔可夫链的自适应估计的算法。该算法利用递归方程来表征马尔可夫链状态的条件分布（给定过去的观察结果）。我们证明，“驱动”算法的过程对于参数的每个固定值都有唯一的不变测度，并且遵循随机近似的常微分方程方法，建立参数估计几乎肯定收敛于相关微分方程的解。通过检查相对于长期平均成本准则的诱导控制马尔可夫过程来分析自适应估计方案的性能。
We investigate an algorithm applied to the adaptive estimation of partially observed finite-state Markov chains. The algorithm utilizes the recursive equation characterizing the conditional distribution of the state of the Markov chain, given the past observations. We show that the process “driving” the algorithm has a unique invariant measure for each fixed value of the parameter, and following the ordinary differential equation method for stochastic approximations, establish almost sure convergence of the parameter estimates to the solutions of an associated differential equation. The performance of the adaptive estimation scheme is analyzed by examining the induced controlled Markov process with respect to a long-run average cost criterion.