New Methodologies for Markov Decision Processes and Stochastic Games Motivated by Inventory Control

库存控制被广泛应用于生产和服务系统以及供应链中，以管理运营并提高效率和可靠性。对几类重要的库存控制问题的分析和优化依赖于马尔可夫决策过程理论，马尔可夫决策过程是研究随机系统序列优化的一个运筹学领域。马尔可夫决策过程理论的两个主要研究方向是：(I)建立最优和近似最优决策的结构；(Ii)发展它们的计算算法。该项目将为马尔可夫决策过程开发新的方法，包括具有不完全信息和风险敏感标准的模型，以及随机博弈。尽管受库存控制问题的推动，该项目的方法进步的潜在应用包括许多应用领域，特别是电力系统的电力储存控制。该项目还将有助于开发科学和工程方面的人力资源。首先，它将支持石溪大学的博士生，包括女学生。其次，它将为研究生和本科生创建研究和教育项目，包括来自未被充分代表的少数群体的学生。该项目将推进两组决策模型的求解方法：马尔可夫决策过程，包括部分可观测的马尔可夫决策过程，以及随机博弈。解决这类问题的最初动机是库存控制应用的启发，本项目也将推动库存控制理论的发展。对于马尔可夫决策过程和部分可观测的马尔可夫决策过程，项目将调查折扣的总成本和平均成本目标。它还将开发在风险、稳健优化、不完全状态信息和不完全了解模型参数的情况下进行决策的方法。具体地说，该项目将在最优性方程和不等的有效性以及最优政策的结构方面建立新的结果。它将开发算法，并研究其对具有经典和非标准标准的问题的收敛和复杂性。对于游戏，该项目将开发一步和连续随机问题的解决方法，这些问题具有完整和不完整的状态观测，可能有无限的回报。

Inventory control is broadly used in production and service systems and in supply chains to manage operations and improve efficiency and reliability. The analysis and optimization of several important classes of inventory control problems relies on the theory of Markov decision processes, an area of operations research dealing with sequential optimization of stochastic systems. The two major research directions in the theory of Markov decision processes are: (i) to establish the structure of optimal and approximately optimal decisions, and (ii) to develop algorithms for their computation. This project will develop new methodologies for Markov decision processes, including models with incomplete information and risk-sensitive criteria, and for stochastic games. Although motivated by inventory control problems, potential applications of this project's methodological advances include many application areas, in particular to the control of electric storage for power systems. The project will also contribute to the development of human resources in science and engineering. First, it will support Ph.D. students at Stony Brook University including female students. Second, it will create research and educational projects for graduate and undergraduate students including students from underrepresented minority groups.This project will advance solution methodologies for two groups of decision making models: Markov decision processes, including partially observable Markov decision processes, and stochastic games. The initial motivation for solving such problems is inspired by inventory control applications, and this project will also advance the inventory control theory. For Markov decision processes and partially observable Markov decision processes, the project will investigate discounted total cost and average cost objectives. It will also develop methodologies for decision making under risk, robust optimization, incomplete state information, and incomplete knowledge of model parameters. Specifically, the project will establish new results on the validity of optimality equations and inequalities and the structure of optimal policies. It will develop algorithms and investigate their convergence and complexity for problems with classic and nonstandard criteria. For games the project will develop solution methodologies for one-step and sequential stochastic problems with complete and incomplete state observations with possibly unbounded payoffs.