Verification Theorems for Markov Decision Processes with Controlled Deterministic Drift and Gradual and Impulsive Controls
Verification Theorems for Markov Decision Processes with Controlled Deterministic Drift and Gradual and Impulsive Controls
复制标题
具有受控确定性漂移以及渐进和脉冲控制的马尔可夫决策过程的验证定理
DOI：
10.1137/1134055
复制
发表时间：
1990
影响因子：
0.6
通讯作者：
A. Yushkevich
中科院分区：
文献类型：
--
作者：
A. Yushkevich
1.导论.在这篇文章中，我们研究了马尔可夫过程的最优控制问题，Kolmogorov曾经建议调用离散干预的机会过程。这些是具有漂移和非扩散跳跃的马尔可夫过程（在戴维斯的术语中，分段确定性马尔可夫过程[1]）。这种类型的受控过程被称为连续时间马尔可夫决策过程。假设该过程可以通过两种类型的控制来作用：1）脉冲控制立即将轨迹转变为新的随机状态，2）渐进控制影响漂移以及在随机时间发生的跳跃的强度和分布。利润泛函是可加性的。对于这类方案，建立了验证定理，允许人们判断给定的函数和马尔可夫策略是否是最大期望收益和一致最优策略。在这些定理中，就像在具有离散时间和无限步数的动态规划中一样，出现了两种类型的条件：1）Bellman方程的类似物以及采取不等式和等式系统形式的守恒条件，2）边界条件，包括已知的均衡条件。在积分形式中，我们得到了充分必要条件，在微分形式中，我们在附加的假设下分别得到了充分必要条件.
1. Introduction. In this article we examine the problem of optimal control for Markov processes of the type that Kolmogorov once suggested calling processes with discrete intervention of chance. These are Markov processes with drift and nonaccumu-lating jumps without diffusion (in the terminology of Davis, piecewise deterministic Markov processes [1]). Controlled processes of such a type have come to be called Markov decision processes with continuous time. It is assumed that the process can be acted upon by controls of two types:1) impulsive controls immediately shifting the trajectory into a new random state, 2) gradual controls influencing the drift and also the intensity and distribution of the jumps occurring at random times. The profit functional is additive. For such schemes, verification theorems are established, permitting one to judge whether the given function and Markov strategy are the maximal expected payoff and a uniformly optimal strategy. In these theorems, as in dynamical programming with discrete time and infinite number of steps, conditions of two types appear: 1) an analogue of the Bellman equation together with theconserving condition taking the form of a system of inequalities and equalities, 2) boundary conditions, including the known equalizing condition. In integral form oneobtains necessary and sufficient conditions and in differential form one obtains separately necessary andsufficient conditions under additional assumptions.