Control of Markov Processes Subject to Qualitative Constraints

离散事件系统（DESs）是根据某些称为事件的离散质变的发生而演变的系统。事件的例子包括排队的顾客到来，计算机程序中算法的终止，通信网络中消息包的丢失，制造系统中机器的故障。在这个建议中，我们考虑由马尔可夫过程建模的随机离散系统，并研究满足定性性质的控制问题。我们概述了一项工作计划，旨在为随机DESs的定性控制制定一个统一的框架。特别是，我们建议研究具有一般定性约束的随机系统的控制，如安全性，非阻塞性，递归性和稳定性；定性约束下的最优控制；完全或部分观察下的控制；使用确定性与随机策略的效果；以及使用平稳与非平稳及渐近平稳控制策略的效果。安全约束被指定为单位区间值向量，作为受控马尔可夫链状态概率分布的上界。非阻塞规范要求达到目标状态集的概率保持在某个最小值之上。递归性或活跃性相当于无限达到目标状态集的概率——通常由一个正常数限制在下面，而收敛性或稳定性要求状态概率分布在有限的步数内进入并保持在一个“安全”集中。

Discrete event systems (DESs) are systems evolving according to the occurrence of certain discrete qualitative changes, called events. Examples of events include the arrival of a customer in a queue, the termination of an algorithm in a computer program, the loss of a message packet in a communication network, the breakdown of a machine in a manufacturing system.In this proposal we consider stochastic DESs modeled by Markov processes, and study control problems that satisfy qualitative properties. We outline a program of work aiming to develop a unified framework for qualitative control of stochastic DESs. In particular, we propose to study the control of stochastic systems with general qualitative constraints such as safety, non-blocking, recurrence, and stability; optimal control subject to qualitative constraints; control under complete or partial observations; effects of using deterministic versus randomized policies; and effects of using stationary versus non-stationary and asymptotically stationary control policies.Safety constraints are specified as unit-interval-valued vectors serving as an upper bound for the state probability distribution of the controlled Markov chain. Non-blocking specifications require that the probability of hitting a target set of states stays above a certain minimum value. Recurrence or liveness amounts to the probability of hitting a target set of states infinitely-often being bounded below by a positive constant, while convergence or stability demands that the state probability distribution enters and stays in a `safe' set within a finite number of steps.