Large scale markov decision processes in healthcare management

本研究的目的是开发一个数学框架，解决大规模的随机决策问题和能力规划的医疗管理。 这些决策包括将患者安排到特定资源以及围绕医院拥挤和病房容量的日常决策。 加拿大医院资源短缺是一个长期且不断恶化的挑战。 因此，有效利用现有资源变得越来越重要。 我们寻求开发一个模型，将同时确定最佳的决策政策和最佳的容量分配。 我们将分析现有快速部署能力（即加班）的影响，并确定快速部署能力和基本能力之间的适当平衡。 马尔可夫决策过程（MDP）是解决随机决策问题的理想方法，但有两个不足之处。 首先，MDP通常按顺序而不是并行地解决决策问题和容量规划问题。 这并没有考虑到这两个问题的相互关联性质。 第二，MDPs在他们能够处理的决策问题的规模上是有限的。 我们寻求应用和改进的近似动态规划（ADP）的方法，允许使用MDP理论的大规模随机决策问题。 这项研究的总体贡献是在医疗管理的实际应用（提高资源利用率和循证能力规划），以及发现的基本见解决策政策和能力规划的相互关联的性质和ADP理论在解决大规模随机决策问题的改进。

The purpose of this research is to develop a mathematical framework for solving large scale stochastic decision problems and capacity planning within healthcare management. Such decisions include the scheduling of patients to a specific resource and the daily decisions around hospital congestion and ward capacity. A shortage of resources in Canadian hospitals is a chronic and steadily worsening challenge. Thus, the efficient use of what resources are available is becoming increasingly critical. We seek to develop a model that will concurrently determine the optimal decision policy and the optimal capacity allocation. We will look to analyse the impact of available surge capacity (i.e. overtime) and determine the appropriate balance between surge and base capacity. Markov decision processes (MDPs) are an ideal method for solving stochastic decision problems but suffer from two deficiencies. First, MDPs generally solve the decision problem and the capacity planning problem in sequence rather than in parallel. This does not take into account the inter-related nature of these two problems. Second, MDPs are limited in the size of decision problem they can handle. We seek to apply and improve the methodologies of approximate dynamic programming (ADP) that allow the use of MDP theory for large scale stochastic decision problems. The overall contributions of this research are both in the practical applications to healthcare management (in improved resource utilization and evidence-based capacity planning) as well as the discovery of fundamental insights into the inter-related nature of decision policies and capacity planning and the improvement of ADP theory in solving large scale stochastic decision problems.