Constrained Optimization of Markov Decision Processes

本项目的主要研究目标是开发新的算法和分析工具来优化和分析广义受控随机系统，这些系统被称为马尔可夫决策过程，当系统性能具有多个准则时。目标是在约束其他标准的情况下对其中一个标准进行优化。本项目将研究以预期总成本和单位时间预期成本为标准的模型。它将集中于寻找大状态空间问题的最优非随机策略，开发多链问题的算法，以及解决未知参数的自适应控制问题。除了一般的马尔可夫决策过程，还将研究三组特殊的问题：所谓的多臂强盗问题，它模拟了许多重要的管理和运筹学问题，呼叫准入和库存控制。如果成功，该项目的结果将为计算重要的随机系统的最优策略提供有效的算法。它们还将描述生产和服务系统的几组数学模型的最优政策结构。目前，高效的算法和结构结果主要是已知的单目标函数问题。然而，现实生活中的应用程序通常要处理多个标准。这个项目将开发数学和工程工具来控制传统计算方法难以处理的大型随机系统。特别是，本课题要研究的多臂强盗问题在调度、医药研究、项目管理、经济等方面都有重要的应用。

The main research objectives of this project are to develop new algorithms and analytical tools for optimization and analysis of broad classes of controlled stochastic systems, known under the name of Markov Decision Processes, when the system performance is characterized by multiple criteria. The objective is to optimize one of the criteria under constraints on other criteria. This project will study models with the criteria of the expected total costs and the expected costs per unit time. It will be focused on finding optimal nonrandomized policies for problems with large state spaces, on developing algorithms for multi-chain problems, and on solving adaptive control problems with unknown parameters. In addition to general Markov Decision Processes, three groups of particular problems will be studied: the so-called multi-armed bandit problems that model many important managerial and operations research problems, call admission, and inventory control. If successful, the results of this project will provide efficient algorithms for computing optimal policies for important classes of stochastic systems. They will also provide descriptions of the structure of optimal policies for several groups of mathematical models of production and service systems. Currently, efficient algorithms and structural results are primarily known for problems with single objective functions. However, real-life applications usually deal with multiple criteria. This project will develop mathematical and engineering tools to control large stochastic systems for which traditional computational methods are intractable. In particular, multi-armed bandit problems to be investigated in this project have important applications to scheduling, pharmaceutical research, project management, and economics.