Nonhomogeneous Markov decision processes in medical decision making

这一建议的目的是建立一个数学框架，当可用行动的集合及其影响随着时间的变化而变化时，进行序贯随机决策。这种情况被建模为非齐次马尔可夫决策过程(MDP)，适用于关于设备更换、设施选址和劳动力规划的决策。我们的动机来自医疗决策问题。例如，当每年在新疗法的研发(R&D)上投入数十亿美元时，患者应该在什么时候开始或更换药物疗法？标准决策模型不考虑这种动态，但最近创建了网站，向医生和他们的患者传播正在进行的临床试验和已经完成的试验结果的信息。据推测，明确考虑未来有效疗法的不确定性的模型将产生与不考虑这种不确定性的模型不同的结果。

The purpose of this proposal is to develop a mathematical framework for sequential stochastic decision making when the set of available actions, and their effects, changes over time. Modeled as nonhomogeneous Markov decision processes (MDPs), such situations apply to decisions about equipment replacement, facility location, and workforce planning. Our motivation comes from medical decision making questions. For example, when should a patient initiate or switch drug therapies when billions of dollars are invested annually in the research and development (R&D) of new therapies? Standard decision models do not consider this dynamic, yet websites have recently been created to disseminate information to physicians and their patients on clinical trials under way and results from trials already completed. Presumably, models that explicitly consider the uncertainty around effective therapies becoming available in the future would yield different results than models that do not consider this uncertainty.