Markov decision processes with a stopping time constraint
Markov decision processes with a stopping time constraint
复制标题
具有停止时间约束的马尔可夫决策过程
DOI：
10.1007/pl00003996
复制
发表时间：
2001
影响因子：
1.2
通讯作者：
M. Horiguchi
中科院分区：
文献类型：
--
作者：
M. Horiguchi
抽象的。本文研究了具有有限状态和行动的停时马氏决策过程在停时τ约束下的最优化问题，使得？对于某个固定的α>0，τ ∈ α。该问题通过停车时间的随机化和占用措施的数学规划来解决。给出了随机停时的另一种表示，称为F-表示，由此引入了马尔可夫或平稳随机停时的概念。我们处理两种类型的占领措施，运行和停止，但停止占领措施被证明是由运行之一表示。我们研究了由不同类别的对政策和随机停止时间实现的运行占用措施的集合的属性。分析了由平稳策略和平稳随机停时对应的运行占用测度所构成的等价数学规划问题，证明了存在一个最优约束的平稳策略和停时对至多在一个状态下要求随机化.
Abstract. In this paper, the optimization problem for a stopped Markov decision process with finite states and actions is considered over stopping times τ constrained so that ?τ≦α for some fixed α>0. The problem is solved through randomization of stopping times and mathematical programming formulation by occupation measures. Another representation, called F-representation, of randomized stopping times is given, by which the concept of Markov or stationary randomized stopping times is introduced. We treat two types of occupation measures, running and stopped, but stopped occupation measure is shown to be expressed by running one. We study the properties of the set of running occupation measures achieved by different classes of pairs of policies and randomized stopping times. Analyzing the equivalent mathematical programming problem formulated by running occupation measures corresponding with stationary policies and stationary randomized stopping times, we prove the existence of an optimal constrained pair of stationary policy and stopping time requiring randomization in at most one state.