Data-driven stochastic dynamic programming approaches for optimal planning of disease screening and chronic disorder management

鉴于向循证和个性化决策的转变，需要强大的数据驱动建模方法来获得针对实际工程问题的个性化最佳干预方案。我们建议开发基于随机动态规划和数据分析的新型工程工具来解决实际的顺序决策问题。虽然拟议的研究将应用于癌症筛查和慢性疾病管理，但重点是开发工程方法。首先，我们提出了一种新的双准则部分可观察马尔可夫决策过程（POMDP）来推导特定筛选问题的最优帕累托有效策略。对多标准pomdp的研究有限；现有的方法是基于状态离散化和机器学习等方法的近似。为了准确地解决我们的POMDP，我们提出了一种新的模型重新表述为约束马尔可夫决策过程（MDP），通过用有限的历史筛选观察值集合取代状态空间。在合理的条件下，这种重新表述提供了一个易于处理的模型，我们的目标是通过结构属性减少状态/动作空间来开发有效的解决程序。利用文献和梅奥诊所的临床数据，我们将应用这一公式来确定结肠直肠癌筛查和监测的帕累托有效政策。所提出的模型可能有助于开发改进和个性化癌症筛查实践的见解，这是一项重大贡献，因为癌症是加拿大的主要死亡原因。所提出的方法也将为其他工程应用提高对pomdp的工程知识。其次，我们提出了一种新的数据驱动方法来模拟不可逆恶化系统（如慢性病）及其管理的进展。许多系统由基于测试/检查结果或技术评估分数的复杂评分系统监控。拟议的数据分析方法将处理来自检查纵向记录的数据，以识别关键事件（收费站），并通过使用包括监督机器学习、分类和预测模型在内的方法，估计这些收费站未来的恶化进程。这些建议的规定性/预测性分析工具的研究结果将用于推导MDP模型，以优化姑息/辅助干预的时间，以减少因病情恶化而产生的负效用。所提出的方法将应用于预测肌萎缩侧索硬化症（ALS）的进展，优化订购辅助设备的时间，并利用罗切斯特梅奥诊所的数据最大限度地提高ALS患者的健康水平。拟议的研究将扩大到考虑更一般的情况。

Given the transformation towards evidence-based and personalized decision making, powerful data-driven modeling approaches are needed to obtain personalized optimal intervention plans for practical engineering problems. We propose developing novel engineering tools based on stochastic dynamic programming and data analytics to address practical sequential decision making problems. Although proposed research will be applied to cancer screening and chronic-disease management, the focus is on developing engineering methodology.First, we propose a novel bicriteria partially observable Markov decision process (POMDP) to derive the optimal Pareto-efficient policies for particular screening problems. There is limited research on multicriteria POMDPs; the existing approaches are approximations based on methods including state discretization and machine learning. To solve our POMDP exactly, we propose a novel reformulation of the model as a constrained Markov decision process (MDP) by replacing the state space with a limited collection of historical screening observations. Under reasonable conditions, this reformulation provides a tractable model for which we aim to develop efficient solution procedures by reducing the state/action space via structural properties. Using clinical data from literature and Mayo Clinic, Rochester-MN, we will apply this formulation to determine the Pareto-efficient policies for colorectal cancer screening and surveillance. The proposed model may help develop insights to improve and personalize cancer screening practices, a significant contribution as cancer is the leading cause of death in Canada. The proposed approach will also improve the engineering knowledge on POMDPs for other engineering applications.Second, we propose a novel data-driven approach for modeling the progression of irreversibly deteriorating systems (e.g., chronic diseases) and their management. Many systems are monitored by complex scoring systems based on test/inspection results or scores from technical assessments. The proposed data analytics approach will process data from longitudinal records of inspections to identify critical events (tollgates) and estimate future progression of deterioration through these tollgates, by using methods including supervised machine learning, classification, and prediction models. The findings from these proposed prescriptive/predictive analytics tools will then be used to derive an MDP model to optimize the timing of palliative/assistive interventions to decrease the disutility due to deterioration. The proposed methodology will be applied to predict amyotrophic lateral sclerosis (ALS) progression, optimize the timing of ordering assistive devices, and maximize wellbeing of ALS patients using data from Mayo Clinic, Rochester.The proposed research will be extended to consider more general settings.