Building a reinforcement learning tool for individually tailoring just-in-time adaptive interventions: Extending the reach of mHealth technology for improved weight loss outcomes

项目摘要 超重与13种癌症有关。这些癌症对黑人和拉丁裔的影响不成比例。 个人以及社会经济地位较低的人，因为超重和肥胖率 在这些群体中的比例更高。行为减肥干预是有效的，但面对面的干预往往 触角较低。随着美国手机拥有量的增加，mHealth技术 承诺覆盖更多的人群，而不是面对面的行为干预。此外，因为他们 个人旅行，并可以实时收集数字信息，mHealth工具使其成为可能 在需要干预的精确时间点与个人进行干预，适时自适应 干预措施(JITAI)。正如目前实施的那样，mHealth JITAI是适应性的，因为干预主义者 可以预先指定决策规则，这些规则会导致由特定用户触发或定制的干预消息 事件。这些实验者指定的决策规则通常基于先前研究的结果， 具体地说，微随机试验提供对mHealth干预消息的顺序测试，以便 根据用户上下文确定消息的因果效果。然而，在此开发的JITAI 方式不能真正地单独定制，因为相同的决策规则平等地应用于每个人 而不考虑有关个人实际如何回应干预消息的信息。迅速地 不断发展的机器学习方法，特别是强化学习(RL)，使改进成为可能 基于当前对JITAI的方法，通过学习每个人的独特响应模式并将 信息转化为后续的、特定于个人的自适应决策规则。然而，少数行为上的 创建了使用RL减肥的mHealth JITAI的干预主义者注意到，在以下方面存在很高的实际障碍 这样做是因为实施RL需要专业知识，而且可能是劳动密集型的。田野 需要一个用户友好的工具来减少这些障碍，以使RL方法得到广泛采用。目标 1是开发Adapt，这是一个迭代地集成实时数据、应用RL算法的工具，以及 进行微随机试验，以优化吉泰减肥决策规则。自适应将会吸引人 实时获取数字健康数据，并使用行为模式和结果进行微随机试验 在正确的时间提供最有效的干预信息，以促进 每一位参与者。目标2是进行为期12周的试点可行性研究，测试Adapt在 减肥干预(NudgeRL)。NudgeRL将建立在团队现有的Jitai，Nush，它确实做到了 不包括RL。该样本将包括20名超重或肥胖的成年人，其中至少50%是 黑人或拉丁裔。尽管ADAPT将被开发来改善减肥干预，但它的广泛使用将 在广泛的健康结果范围内产生更有效和有效的JITAI，从而导致较低的 由于健康行为的广泛改善，癌症和其他疾病的负担。

Project Summary Excess weight is associated with 13 types of cancer. These cancers disproportionately affect Black and Latinx individuals, as well as those with lower socioeconomic status, because overweight and obesity incidence is higher in these groups. Behavioral weight loss interventions are effective, but in-person interventions tend to have low reach. As mobile phone ownership is increasing in the United States, mHealth technology holds promise for reaching a larger population than in-person behavioral interventions. Furthermore, because they travel with individuals and can collect digital information in real-time, mHealth tools make it possible to intervene with individuals at the precise point when the interventions are needed with just-in-time adaptive interventions (JITAIs). As currently implemented, mHealth JITAIs are adaptive in the sense that interventionists can specify decision rules a priori that result in intervention messages that are triggered or tailored by certain events. These experimenter-specified decision rules are generally based upon results of prior studies, specifically micro-randomized trials that provide sequential tests of mHealth intervention messages in order to determine causal effects of messages conditional on user context. However, JITAIs that are developed in this manner cannot be truly individually tailored because the same decision rules are equally applied to everybody without regard to information about how individuals actually respond to intervention messages. Rapidly evolving machine learning methods, specifically reinforcement learning (RL), makes it possible to improve upon the current approach to JITAIs by learning each person's unique response patterns and integrating this information into subsequent, person-specific, adaptive decision rules. However, the few behavioral interventionists who have created mHealth JITAIs for weight loss using RL have noted high practical barriers to doing so because implementation of RL requires specialized expertise and can be labor intensive. The field needs a user-friendly tool to reduce these barriers in order for RL methodology to become widely adopted. Aim 1 is to develop Adapt, a tool that iteratively integrates real-time data, applies RL algorithms, and performs micro-randomized trials to optimize JITAI decision rules for weight loss. Adapt will pull in digital health data in real-time and conduct micro-randomized trials using behavioral patterns and outcomes to arrive at the most efficacious intervention message, delivered at the right time, for promoting weight loss in each participant. Aim 2 is to conduct a 12-week pilot feasibility study testing usability of Adapt in a weight loss intervention (NudgeRL). NudgeRL will build upon the team's existing JITAI, Nudge, which did not incorporate RL. The sample will consist of 20 adults with overweight or obesity, at least 50% of whom are Black or Latinx. Although Adapt will be developed to improve weight loss interventions, its widespread use will result in more efficient and efficacious JITAIs across a broad range of health outcomes, resulting in a lower burden of cancer and other disease due to a wide spectrum of improved health behaviors.