Mathematical Sciences: Topics in Markov Decision Processes

数学科学：马尔可夫决策过程主题

• 批准号: 9404177
• 负责人: Alexander Yushkevich
• 金额: $ 4万
• 依托单位: University of North Carolina at Charlotte
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 1994
• 资助国家: 美国
• 起止时间: 1994-10-15 至 1996-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9404177&HistoricalAwards=false
• 关键词: Mathematical; Sciences; Topics; Markov; Decision

9404177 Yushkevich The Theory of Markov Decision Processes (MDP's) is an important tool in operations research, management sciences, and numerical solution of continuous time stochastic control problems. Sensitive criteria in MDP's permit adjustments for the underselectiveness of the average per unit time reward criterion in cases where initial stages are important and they also provide deeper insight into the nature of optimality conditions and equations. Up to now, sensitive criteria have been studied only for MDP's with finite or countable state spaces. Models with a continuous state space are more relevant in many applications, for instance, when there are incomplete observations. The purpose of the proposed research is to extend the theory of sensitive criteria to MDP's with a Borel state space. First of all models with an absolutely continuous transition function will be studied. For such models, we expect to obtain (by means of limit theorems for general state space Markov chains, Laurent expansions of resolvents, and, especially, new techniques for aggregation and compactification of the set of feedback controls) the following results: 1)validity of the lexicographical optimality equation, 2) existence of deterministic stationary sensitively optimal policies, and 3) effectiveness of a lexicographical policy improvement algorithm to get such a policy. Markov Decision Processes provide important tools for the analysis of many control and management optimization problems in which randomness plays a significant role. They are often used to optimize an operation's resource allocations among competing requirements (for example, inventory costs in maintaining a certain number of spare parts for a system versus down time costs for the system versus costs of repairing defective parts of the system in a given amount of time). In many of these problems, the long-run average cost (or reward) is the natural objective function to optimize. However, long-run average criteria are in sensitive to costs or rewards associated with initial stages, which, under certain circumstances, can lead to nonoptimal policies. Sensitive criteria have only been successfully analyzed for the case when the possible states of the system are finite or countable. There are many realistic situations, however, when the appropriate state space is a continuum, for example, when incomplete observations are involved, which is often the case. It is the purpose of this research to extend the theory of sensitive criteria to Markov Decision Processes with Borel state spaces (which are general enough to include all known situations of importance).

9404177 尤什克维奇 马尔可夫决策过程理论（MDP）是运筹学、管理科学和连续时间随机控制问题数值解的重要工具。在MDP的敏感标准允许调整的平均每单位时间奖励标准的选择性不足的情况下，初始阶段是重要的，他们也提供了更深入的了解最优性条件和方程的性质。到目前为止，敏感性准则只研究了有限或可数状态空间的MDP。具有连续状态空间的模型在许多应用中更相关，例如，当存在不完整的观测时。所提出的研究的目的是将敏感准则理论扩展到具有Borel状态空间的MDP。首先研究具有绝对连续转移函数的模型。 对于这样的模型，我们期望获得（通过一般状态空间马尔可夫链的极限定理，预解式的Laurent展开，特别是反馈控制集的聚集和紧化的新技术）得到以下结果：1）字典序最优性方程的有效性，2）确定性平稳灵敏最优策略的存在性，以及3）词典策略改进算法获得这样的策略的有效性。 马尔可夫决策过程为分析许多控制和管理优化问题提供了重要的工具，其中随机性起着重要的作用。它们通常用于优化操作的资源分配之间的竞争的要求（例如，库存成本在维护一定数量的备件的系统与系统的停机时间成本与成本修复系统的缺陷部件在给定的时间量）。在许多这些问题中，长期平均成本（或回报）是优化的自然目标函数。然而，长期平均标准对与初始阶段相关的成本或回报敏感，在某些情况下，这可能会导致非最优政策。灵敏度准则只成功地分析了系统的可能状态是有限或可数的情况。然而，有许多现实的情况下，当适当的状态空间是一个连续体，例如，当不完整的观察，这是经常发生的情况。本研究的目的是将敏感准则理论推广到具有Borel状态空间的马尔可夫决策过程（该状态空间足够一般，可以包括所有已知的重要情况）。