New Methodologies for Markov Decision Processes and Stochastic Games Motivated by Inventory Control

库存控制广泛应用于生产和服务系统以及供应链中，以管理运营并提高效率和可靠性。 几类重要的库存控制问题的分析和优化依赖于马尔可夫决策过程理论，这是处理随机系统顺序优化的运筹学领域。 马尔可夫决策过程理论的两个主要研究方向是：（i）建立最优和近似最优决策的结构，以及（ii）开发其计算算法。该项目将为马尔可夫决策过程开发新的方法，包括具有不完整信息和风险敏感标准的模型以及随机博弈。尽管受到库存控制问题的推动，该项目方法论进步的潜在应用包括许多应用领域，特别是电力系统蓄电控制。该项目还将有助于科学和工程人力资源的开发。首先，它将支持博士学位。石溪大学的学生，包括女学生。其次，它将为研究生和本科生创建研究和教育项目，包括来自代表性不足的少数群体的学生。该项目将推进两组决策模型的解决方法：马尔可夫决策过程，包括部分可观察的马尔可夫决策过程和随机博弈。 解决此类问题的最初动机是受到库存控制应用的启发，该项目也将推进库存控制理论。 对于马尔可夫决策过程和部分可观察马尔可夫决策过程，该项目将研究贴现总成本和平均成本目标。它还将开发在风险、鲁棒优化、不完整状态信息和模型参数不完整知识下进行决策的方法。 具体来说，该项目将在最优方程和不等式的有效性以及最优政策的结构方面建立新的结果。它将开发算法并研究其针对经典和非标准标准问题的收敛性和复杂性。对于游戏，该项目将为具有完整和不完整状态观察且可能具有无限回报的一步和顺序随机问题开发解决方案。

Inventory control is broadly used in production and service systems and in supply chains to manage operations and improve efficiency and reliability. The analysis and optimization of several important classes of inventory control problems relies on the theory of Markov decision processes, an area of operations research dealing with sequential optimization of stochastic systems. The two major research directions in the theory of Markov decision processes are: (i) to establish the structure of optimal and approximately optimal decisions, and (ii) to develop algorithms for their computation. This project will develop new methodologies for Markov decision processes, including models with incomplete information and risk-sensitive criteria, and for stochastic games. Although motivated by inventory control problems, potential applications of this project's methodological advances include many application areas, in particular to the control of electric storage for power systems. The project will also contribute to the development of human resources in science and engineering. First, it will support Ph.D. students at Stony Brook University including female students. Second, it will create research and educational projects for graduate and undergraduate students including students from underrepresented minority groups.This project will advance solution methodologies for two groups of decision making models: Markov decision processes, including partially observable Markov decision processes, and stochastic games. The initial motivation for solving such problems is inspired by inventory control applications, and this project will also advance the inventory control theory. For Markov decision processes and partially observable Markov decision processes, the project will investigate discounted total cost and average cost objectives. It will also develop methodologies for decision making under risk, robust optimization, incomplete state information, and incomplete knowledge of model parameters. Specifically, the project will establish new results on the validity of optimality equations and inequalities and the structure of optimal policies. It will develop algorithms and investigate their convergence and complexity for problems with classic and nonstandard criteria. For games the project will develop solution methodologies for one-step and sequential stochastic problems with complete and incomplete state observations with possibly unbounded payoffs.