Tools for the Precision Medicine Era: How to Develop Highly Personalized Treatment Recommendations From Cohort and Registry Data Using Q-Learning
Tools for the Precision Medicine Era: How to Develop Highly Personalized Treatment Recommendations From Cohort and Registry Data Using Q-Learning
复制标题
DOI：
10.1093/aje/kwx027
复制
发表时间：
2017-07-15
影响因子：
5
通讯作者：
Moodie, Erica E. M.
中科院分区：
文献类型：
--
作者：
Krakow, Elizabeth F.;Hemmer, Michael;Moodie, Erica E. M.
Q学习是一种强化学习方法，它采用向后逐阶段估计来识别最大化一些长期奖励的动作序列。该方法可应用于连续多分配随机试验，以制定个性化的适应性治疗策略（ATS）-纵向实践指南，高度适应个体患者随时间变化的属性。有时，选择哪些ATS纳入序贯多分配随机试验（或随机对照试验）的基础可能不充分。非随机化数据源可能会为ATS的初始设计提供信息，随后可能会进行前瞻性验证。本文在分析了现有文献的基础上，我们通过国际血液和骨髓移植研究中心注册处的一项案例研究，说明了使用非随机数据进行这一目的所面临的挑战（1995-2007）旨在1）确定用于移植物抗宿主病预防和难治性移植物抗宿主病的治疗类别顺序是否与生存率改善相关，以及2）确定供者和患者因素，以指导随着时间的推移个性化免疫抑制剂的选择。我们讨论了如何沟通的潜在好处来自以下ATS在人口和亚组水平，以及如何评估其鲁棒性建模假设。该工作示例可以作为肿瘤学和其他需要顺序治疗决策的领域中的登记和队列开发ATS的模型。
Q-learning is a method of reinforcement learning that employs backwards stagewise estimation to identify sequences of actions that maximize some long-term reward. The method can be applied to sequential multiple-assignment randomized trials to develop personalized adaptive treatment strategies (ATSs)-longitudinal practice guidelines highly tailored to time-varying attributes of individual patients. Sometimes, the basis for choosing which ATSs to include in a sequential multiple-assignment randomized trial (or randomized controlled trial) may be inadequate. Nonrandomized data sources may inform the initial design of ATSs, which could later be prospectively validated. In this paper, we illustrate challenges involved in using nonrandomized data for this purpose with a case study from the Center for International Blood and Marrow Transplant Research registry (1995-2007) aimed at 1) determining whether the sequence of therapeutic classes used in graft-versus-host disease prophylaxis and in refractory graft-versus-host disease is associated with improved survival and 2) identifying donor and patient factors with which to guide individualized immunosuppressant selections over time. We discuss how to communicate the potential benefit derived from following an ATS at the population and subgroup levels and how to evaluate its robustness to modeling assumptions. This worked example may serve as a model for developing ATSs from registries and cohorts in oncology and other fields requiring sequential treatment decisions.