Nonstationary Markov decision problems with converging parameters
Nonstationary Markov decision problems with converging parameters
复制标题
具有收敛参数的非平稳马尔可夫决策问题
DOI：
10.1007/bf00935474
复制
发表时间：
1981
影响因子：
1.9
通讯作者：
P. Schweitzer
中科院分区：
文献类型：
--
作者：
A. Federgruen;P. Schweitzer
本文考虑马尔可夫决策问题的解决方案，其参数只能通过近似方案获得，或者在计算上最好近似的参数，而不是采用精确的算法来计算。此外，它示出了一个修改的值迭代方法，可以采用，无论是贴现版本和未贴现版本的模型，以解决最优性方程，并找到最优的政策。在这两种情况下，收敛速度是确定的。作为一个副结果，我们刻画了一个几何收敛的马尔可夫矩阵序列的后向积的渐近行为。
This paper considers the solution of Markov decision problems whose parameters can be obtained only via approximating schemes, or where it is computationally preferable to approximate the parameters, rather than employing exact algorithms for their computation.Various models are presented in which this situation occurs. Furthermore, it is shown that a modified value-iteration method may be employed, both for the discounted version and for the undiscounted version of the model, in order to solve the optimality equation and to find optimal policies. In both cases, the convergence rate is determined.As a side result, we characterize the asymptotic behavior of backward products of a geometrically convergent sequence of Markov matrices.