mDOT TR&D2 (Optimization): Dynamic Optimization of Continuously Adapting mHealth Interventions via Prudent, Statistically Efficient, and Coherent Reinforcement Learning

项目负责人：Murphy，Susan首席研究员：Kumar，Santosh Tr&d2：通过审慎的、 统计高效、连贯的强化学习 领衔：苏珊·墨菲博士，哈佛大学；10%努力(1.2厘米) 摘要：移动健康中心发现、优化和翻译时间精确干预措施 (MDOT中心)将实现一种时间精准医学的新范式，以维持健康和管理 慢性病带来的负担越来越大。MDOT中心将开发和传播方法、工具、 和基础设施，为研究人员进行发现、优化和翻译临时- 精确的移动健康干预。这种干预，当动态地个性化到每一时刻时 每个人的生物-心理-社会-环境背景，将在 医疗保健，使患者能够开始并维持直接 管理、治疗，在某些情况下甚至阻止医疗条件的发展。有条理的 围绕着三个技术研发项目，MDOT代表着一个独特的国家 将开发多种方法和技术创新并支持其翻译的资源 以易于部署的可穿戴设备、应用程序的形式由mHealth社区进行研究和实践 可穿戴设备和智能手机，以及配套的mHealth云系统，都是开源的。 技术研究和开发项目2(tr&d2)将解决当前在线的三个关键限制 强化学习(RL)应用于对个人的个性化移动干预。其中的两个 局限性与需要提高疗效和减少延迟干预的负面负担有关。 导致脱节。第三，着眼于涉及多种干预的个性化的未来需求 每个组件都以不同的时间尺度运行。特别是，我们将容纳无时无刻不在的移动设备 通过开发RL算法之间的方法连续体来应对用户脱离的健康挑战 忽略延迟干预效果和尝试捕获噪声延迟干预效果的RL算法 在更遥远的未来。其次，我们将通过最佳方式提高个性化发生的速度 利用跨时间和跨用户的数据，更快地对每个用户进行个性化干预。第三, 我们将开发第一个RL方法，以一致地整体地个性化多个干预组件。 此外，为加强影响和传播，将与三个组织密切合作制定这些方法。 强调模型可解释性的协作项目。我们将提供这两个服务项目和 更广泛的研究社区，拥有开源软件工具和系统，包括智能手机和 用于在线个性化的云计算组件。R&D2将协同工作，与 其他研发项目，培训和传播核心，以及行政核心，以最大限度地提高社会 Tr&d2技术的影响。 0

Project Lead: Murphy, Susan Principal Investigator: Kumar, Santosh TR&D2: Dynamic Optimization of Continuously Adapting mHealth Interventions via Prudent, Statistically Efficient, and Coherent Reinforcement Learning Lead: Dr. Susan Murphy, Harvard University; 10% effort (1.2CM) Abstract: The mHealth Center for Discovery, Optimization & Translation of Temporally-Precise Interventions (the mDOT Center) will enable a new paradigm of temporally-precise medicine to maintain health and manage the growing burden of chronic diseases. The mDOT Center will develop and disseminate the methods, tools, and infrastructure necessary for researchers to pursue the discovery, optimization and translation of temporally- precise mHealth interventions. Such interventions, when dynamically personalized to the moment-to-moment biopsychosocial-environmental context of each individual, will precipitate a much-needed transformation in healthcare by enabling patients to initiate and sustain the healthy lifestyle choices necessary for directly managing, treating, and in some cases even preventing the development of medical conditions. Organized around three Technology Research & Development (TR&D) projects, mDOT represents a unique national resource that will develop multiple methodological and technological innovations and support their translation into research and practice by the mHealth community in the form of easily deployable wearables, apps for wearables and smartphones, and a companion mHealth cloud system, all open-source. Technology Research and Development project 2 (TR&D2) will address three key limitations of current online reinforcement learning (RL) when applied to personalize mobile interventions to individuals. Two of these limitations are related to the need to increase efficacy and reduce negative delayed intervention burden effects leading to disengagement. The third looks to future needs involving the personalization of multiple intervention components each operating at a different time scale. In particular, we will accommodate the ever-present mobile health challenge of user disengagement by developing a continuum of approaches between RL algorithms that ignore delayed intervention effects and RL algorithms that attempt to capture noisy delayed intervention effects over a more distant future. Second, we will increase the rate at which personalization occurs via optimally leveraging data across time and across users to more quickly personalize the interventions to each user. Third, we will develop the first RL approaches to coherently personalize multiple intervention components holistically. In addition, to enhance impact and dissemination, the methods will be developed in close collaboration with three collaborative projects with an emphasis on model interpretability. We will provide the two service projects and the broader research community with open-source software tools and systems consisting of smartphone and cloud computing components for online personalization. TR&D2 will synergistically work in partnership with the other TR&D projects, the Training and Dissemination Core, and the Administration Core to maximize the societal impact of TR&D2 technologies. 0