Nonhomogeneous Markov decision processes in medical decision making

这一建议的目的是开发一个数学框架，当一组可用的行动及其影响随时间变化时，进行顺序随机决策。将这种情况建模为非齐次马尔可夫决策过程（mdp），适用于有关设备更换、设施位置和劳动力计划的决策。我们的动力来自医疗决策问题。例如，当每年投入数十亿美元用于新疗法的研发（R&D）时，患者应该何时开始或转换药物治疗？标准的决策模型没有考虑到这种动态，然而，最近已经创建了网站，向医生和他们的患者传播正在进行的临床试验和已经完成的试验结果的信息。据推测，明确考虑到未来有效疗法的不确定性的模型与不考虑这种不确定性的模型会产生不同的结果。

The purpose of this proposal is to develop a mathematical framework for sequential stochastic decision making when the set of available actions, and their effects, changes over time. Modeled as nonhomogeneous Markov decision processes (MDPs), such situations apply to decisions about equipment replacement, facility location, and workforce planning. Our motivation comes from medical decision making questions. For example, when should a patient initiate or switch drug therapies when billions of dollars are invested annually in the research and development (R&D) of new therapies? Standard decision models do not consider this dynamic, yet websites have recently been created to disseminate information to physicians and their patients on clinical trials under way and results from trials already completed. Presumably, models that explicitly consider the uncertainty around effective therapies becoming available in the future would yield different results than models that do not consider this uncertainty.