Building a reinforcement learning tool for individually tailoring just-in-time adaptive interventions: Extending the reach of mHealth technology for improved weight loss outcomes

项目摘要 超重与13种癌症有关。这些癌症不成比例地影响黑人和拉丁裔 个人，以及那些社会经济地位较低的人，因为超重和肥胖的发病率是 在这些群体中，行为减肥干预是有效的，但在人的干预往往 低到达。随着美国移动的手机拥有量的增加， 比面对面的行为干预更有希望接触到更多的人群。此外，因为他们 与个人一起旅行，并可以实时收集数字信息，mHealth工具使之成为可能， 在需要干预的精确点上对个人进行干预， （JITAIs）。正如目前实施的，mHealth JITAIs是自适应的，因为干预者 可以先验地指定决策规则，这些决策规则导致由某些 事件这些实验者指定的决策规则通常基于先前研究的结果， 特别是微随机试验，提供移动健康干预信息的顺序测试， 根据用户上下文确定消息的因果关系。然而，在这种情况下开发的JITAI 因为同样的决策规则同样适用于每个人， 而不考虑关于个人如何实际响应干预消息的信息。迅速 不断发展的机器学习方法，特别是强化学习（RL），使得提高 通过学习每个人独特的反应模式， 将信息转化为后续的、特定于个人的、自适应的决策规则。然而，少数行为 使用RL创建mHealth JITAIs减肥的干预者注意到， 这样做是因为RL的实现需要专业知识，并且可能是劳动密集型的。领域 需要一个用户友好的工具来减少这些障碍，以便RL方法被广泛采用。目的 1是开发Adapt，这是一种迭代集成实时数据、应用RL算法的工具，并且 进行微随机试验，以优化吉泰减肥决策规则。Adapt会拉进来 实时数字健康数据，并使用行为模式和结果进行微随机试验， 达到最有效的干预信息，在正确的时间传递，以促进减肥， 每个参与者。目标2是进行一项为期12周的试点可行性研究， 减肥干预（NudgeRL）。NudgeRL将建立在团队现有的JITAI Nudge的基础上， 不包含RL。该样本将由20名超重或肥胖的成年人组成，其中至少50%是 黑人或拉丁裔虽然Adapt将被开发用于改善减肥干预措施，但其广泛使用将 在广泛的健康结果中产生更有效和更有效的JITAI，从而降低 癌症和其他疾病的负担，由于广泛的改善健康行为。

Project Summary Excess weight is associated with 13 types of cancer. These cancers disproportionately affect Black and Latinx individuals, as well as those with lower socioeconomic status, because overweight and obesity incidence is higher in these groups. Behavioral weight loss interventions are effective, but in-person interventions tend to have low reach. As mobile phone ownership is increasing in the United States, mHealth technology holds promise for reaching a larger population than in-person behavioral interventions. Furthermore, because they travel with individuals and can collect digital information in real-time, mHealth tools make it possible to intervene with individuals at the precise point when the interventions are needed with just-in-time adaptive interventions (JITAIs). As currently implemented, mHealth JITAIs are adaptive in the sense that interventionists can specify decision rules a priori that result in intervention messages that are triggered or tailored by certain events. These experimenter-specified decision rules are generally based upon results of prior studies, specifically micro-randomized trials that provide sequential tests of mHealth intervention messages in order to determine causal effects of messages conditional on user context. However, JITAIs that are developed in this manner cannot be truly individually tailored because the same decision rules are equally applied to everybody without regard to information about how individuals actually respond to intervention messages. Rapidly evolving machine learning methods, specifically reinforcement learning (RL), makes it possible to improve upon the current approach to JITAIs by learning each person's unique response patterns and integrating this information into subsequent, person-specific, adaptive decision rules. However, the few behavioral interventionists who have created mHealth JITAIs for weight loss using RL have noted high practical barriers to doing so because implementation of RL requires specialized expertise and can be labor intensive. The field needs a user-friendly tool to reduce these barriers in order for RL methodology to become widely adopted. Aim 1 is to develop Adapt, a tool that iteratively integrates real-time data, applies RL algorithms, and performs micro-randomized trials to optimize JITAI decision rules for weight loss. Adapt will pull in digital health data in real-time and conduct micro-randomized trials using behavioral patterns and outcomes to arrive at the most efficacious intervention message, delivered at the right time, for promoting weight loss in each participant. Aim 2 is to conduct a 12-week pilot feasibility study testing usability of Adapt in a weight loss intervention (NudgeRL). NudgeRL will build upon the team's existing JITAI, Nudge, which did not incorporate RL. The sample will consist of 20 adults with overweight or obesity, at least 50% of whom are Black or Latinx. Although Adapt will be developed to improve weight loss interventions, its widespread use will result in more efficient and efficacious JITAIs across a broad range of health outcomes, resulting in a lower burden of cancer and other disease due to a wide spectrum of improved health behaviors.