Nonhomogeneous Markov decision processes in medical decision making

医疗决策中的非齐次马尔可夫决策过程

• 批准号: 341415-2007
• 负责人: Shechter, Steven
• 金额: $ 1.24万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2007
• 资助国家: 加拿大
• 起止时间: 2007-01-01 至 2008-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=365551
• 关键词: Nonhomogeneous; Markov; decision; processes; medical

The purpose of this proposal is to develop a mathematical framework for sequential stochastic decision making when the set of available actions, and their effects, changes over time.  Modeled as nonhomogeneous Markov decision processes (MDPs), such situations apply to decisions about equipment replacement, facility location, and workforce planning.  Our motivation comes from medical decision making questions.  For example, when should a patient initiate or switch drug therapies when billions of dollars are invested annually in the research and development (R&D) of new therapies? Standard decision models do not consider this dynamic, yet websites have recently been created to disseminate information to physicians and their patients on clinical trials under way and results from trials already completed. Presumably, models that explicitly consider the uncertainty around effective therapies becoming available in the future would yield different results than models that do not consider this uncertainty.     The proposed research aims to quantify differences between models that do not consider evolving actions and effects, and models that do consider these dynamics.  In doing this, we will develop methods for estimating model parameters that describe R&D, such as probabilities of therapies becoming available at different times in the future, along with their effects.  Other methodological contributions will include discovering structural properties of optimal solutions and finding efficient solution algorithms. The overall contributions of this research are both in the practical applications to medical decision making problems as well as the discovery of fundamental insights when considering time-varying action sets in MDPs.

本文的目的是建立一个数学框架，当可用的行动集及其影响随时间变化时，进行顺序随机决策。建模为非齐次马尔可夫决策过程（MDP），这种情况适用于有关设备更换，设施位置和劳动力规划的决策。我们的动机来自医疗决策问题。例如，当每年数十亿美元投资于新疗法的研究和开发（R&D）时，患者应该何时开始或转换药物治疗？标准决策模型不考虑这种动态，但最近创建了网站，向医生及其患者传播正在进行的临床试验和已完成试验结果的信息。据推测，明确考虑未来有效疗法的不确定性的模型将产生与不考虑这种不确定性的模型不同的结果。 该研究旨在量化不考虑不断变化的作用和影响的模型与考虑这些动态的模型之间的差异。在此过程中，我们将开发用于估计描述研发的模型参数的方法，例如在未来不同时间可用的疗法的概率，沿着它们的影响。其他方法学贡献将包括发现最优解的结构特性和找到有效的解算法。这项研究的总体贡献是在实际应用中的医疗决策问题，以及发现的基本见解时，考虑随时间变化的动作集在MDPs。