A sensitivity-based construction approach to sample-path variance minimization of Markov decision processes
A sensitivity-based construction approach to sample-path variance minimization of Markov decision processes
复制标题
DOI：
--
复制
发表时间：
2012-11
期刊：
影响因子：
--
通讯作者：
Yonghao Huang;Xi Chen
中科院分区：
文献类型：
--
作者：
Yonghao Huang;Xi Chen
本文以马尔可夫决策过程的长期平均性能为主要准则，研究了沿样本路径的极限平均方差作为马尔可夫决策过程的次要准则。采用基于灵敏度的方法，直观地构造了不同策略下样本路径方差的差分公式。由此，可以很容易地推导出样本路径方差最优性的充分条件。本文将基于灵敏度的构造方法扩展到具有非标准性能准则的马尔可夫决策过程。与纯数学验证方法相比，基于灵敏度的构造方法更直观，并提供了对马尔可夫决策过程样本路径结构的见解。
We study the limiting average variance along the sample path as the secondary criterion for Markov decision processes, with the long-run average performance as the primary criterion. By applying the sensitivity-based approach, we intuitively construct the difference formula for the sample-path variance under different policies. Thereby, a sufficient condition for the sample-path variance optimality can be easily derived. This work extends the sensitivity-based construction approach to the Markov decision processes with the nonstandard performance criterion. Compared with the pure mathematical verification, the sensitivity-based construction approach shows more intuition and provides insights on the sample-path structure of Markov decision processes.