Control of Markov Processes Subject to Qualitative Constraints

离散事件系统（DES）是根据某些离散的定性变化（称为事件）的发生而演化的系统。 事件的例子包括到达的客户在一个队列中，在计算机程序中的算法的终止，在通信网络中的消息包的丢失，在制造系统中的机器的故障，在这个建议中，我们考虑随机DES建模的马尔可夫过程，并研究控制问题，满足定性性质。 我们概述了一个工作计划，旨在制定一个统一的框架，定性控制随机DES。 特别是，我们建议研究具有一般定性约束的随机系统的控制，如安全性，非阻塞性，递归性和稳定性;定性约束下的最优控制;完全或部分观测下的控制;使用确定性与随机策略的影响;以及使用平稳与非平稳和渐近平稳控制策略的效果。安全约束被指定为单位间隔-值向量作为受控马尔可夫链的状态概率分布的上界。 非阻塞规范要求命中目标状态集的概率保持在某个最小值以上。 递归或活性的数额的概率击中一组目标的状态无限，往往是有界以下的一个正常数，而收敛或稳定性要求的状态概率分布进入并停留在一个“安全”的设置在有限数量的步骤。

Discrete event systems (DESs) are systems evolving according to the occurrence of certain discrete qualitative changes, called events. Examples of events include the arrival of a customer in a queue, the termination of an algorithm in a computer program, the loss of a message packet in a communication network, the breakdown of a machine in a manufacturing system.In this proposal we consider stochastic DESs modeled by Markov processes, and study control problems that satisfy qualitative properties. We outline a program of work aiming to develop a unified framework for qualitative control of stochastic DESs. In particular, we propose to study the control of stochastic systems with general qualitative constraints such as safety, non-blocking, recurrence, and stability; optimal control subject to qualitative constraints; control under complete or partial observations; effects of using deterministic versus randomized policies; and effects of using stationary versus non-stationary and asymptotically stationary control policies.Safety constraints are specified as unit-interval-valued vectors serving as an upper bound for the state probability distribution of the controlled Markov chain. Non-blocking specifications require that the probability of hitting a target set of states stays above a certain minimum value. Recurrence or liveness amounts to the probability of hitting a target set of states infinitely-often being bounded below by a positive constant, while convergence or stability demands that the state probability distribution enters and stays in a `safe' set within a finite number of steps.