Building a reinforcement learning tool for individually tailoring just-in-time adaptive interventions: Extending the reach of mHealth technology for improved weight loss outcomes

构建强化学习工具，用于个性化定制及时适应性干预措施：扩大移动医疗技术的覆盖范围，以改善减肥效果

• 批准号: 10195835
• 负责人: Kimberly Brooke Tompkins Nezami
• 金额: $ 41.09万
• 依托单位: UNIV OF NORTH CAROLINA CHAPEL HILL
• 依托单位国家: 美国
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-05-01 至 2024-04-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10195835
• 关键词: Address; Adopted; Adult; Affect; African American; Behavior; Behavior Therapy; Behavioral; Body Weight decreased; Cancer Burden; Cancer Control; Car Phone; Cellular Phone; Clinical; Data; Development; Diet; Disease; Effectiveness; Event; Feasibility Studies; Goals; Health; Health behavior; Hispanics; Incidence; Individual; Intervention; Latinx; Lead; Learning; Link; Machine Learning; Malignant Neoplasms; Mediating; Methodology; Monitor; Obesity; Obesity associated disease; Online Systems; Outcome; Overweight; Ownership; Participant; Pattern; Persons; Physical activity; Population; Positive Reinforcements; Psychological reinforcement; Randomized; Rewards; Risk Factors; Sampling; Specific qualifier value; Techniques; Technology; Testing; Time; Travel; United States; Update; Weight; Weight maintenance regimen; Work; adaptive intervention; base; behavior change; cancer health disparity; cancer prevention; cancer type; clinically significant; cost; digital; digital health; effective intervention; efficacious intervention; fitbit; follow-up; health data; health disparity; improved; insight; learning algorithm; low socioeconomic status; mHealth; machine learning method; obesity risk; programs; racial and ethnic; randomized trial; response; socioeconomics; time use; tool; usability; user-friendly; web-based tool; weight loss intervention

Project Summary Excess weight is associated with 13 types of cancer. These cancers disproportionately affect Black and Latinx individuals, as well as those with lower socioeconomic status, because overweight and obesity incidence is higher in these groups. Behavioral weight loss interventions are effective, but in-person interventions tend to have low reach. As mobile phone ownership is increasing in the United States, mHealth technology holds promise for reaching a larger population than in-person behavioral interventions. Furthermore, because they travel with individuals and can collect digital information in real-time, mHealth tools make it possible to intervene with individuals at the precise point when the interventions are needed with just-in-time adaptive interventions (JITAIs). As currently implemented, mHealth JITAIs are adaptive in the sense that interventionists can specify decision rules a priori that result in intervention messages that are triggered or tailored by certain events. These experimenter-specified decision rules are generally based upon results of prior studies, specifically micro-randomized trials that provide sequential tests of mHealth intervention messages in order to determine causal effects of messages conditional on user context. However, JITAIs that are developed in this manner cannot be truly individually tailored because the same decision rules are equally applied to everybody without regard to information about how individuals actually respond to intervention messages. Rapidly evolving machine learning methods, specifically reinforcement learning (RL), makes it possible to improve upon the current approach to JITAIs by learning each person's unique response patterns and integrating this information into subsequent, person-specific, adaptive decision rules. However, the few behavioral interventionists who have created mHealth JITAIs for weight loss using RL have noted high practical barriers to doing so because implementation of RL requires specialized expertise and can be labor intensive. The field needs a user-friendly tool to reduce these barriers in order for RL methodology to become widely adopted. Aim 1 is to develop Adapt, a tool that iteratively integrates real-time data, applies RL algorithms, and performs micro-randomized trials to optimize JITAI decision rules for weight loss. Adapt will pull in digital health data in real-time and conduct micro-randomized trials using behavioral patterns and outcomes to arrive at the most efficacious intervention message, delivered at the right time, for promoting weight loss in each participant. Aim 2 is to conduct a 12-week pilot feasibility study testing usability of Adapt in a weight loss intervention (NudgeRL). NudgeRL will build upon the team's existing JITAI, Nudge, which did not incorporate RL. The sample will consist of 20 adults with overweight or obesity, at least 50% of whom are Black or Latinx. Although Adapt will be developed to improve weight loss interventions, its widespread use will result in more efficient and efficacious JITAIs across a broad range of health outcomes, resulting in a lower burden of cancer and other disease due to a wide spectrum of improved health behaviors.

项目摘要 超重与13种癌症有关。这些癌症不成比例地影响黑人和拉丁裔 个人，以及那些社会经济地位较低的人，因为超重和肥胖的发病率是 在这些群体中，行为减肥干预是有效的，但在人的干预往往 低到达。随着美国移动的手机拥有量的增加， 比面对面的行为干预更有希望接触到更多的人群。此外，因为他们 与个人一起旅行，并可以实时收集数字信息，mHealth工具使之成为可能， 在需要干预的精确点上对个人进行干预， （JITAIs）。正如目前实施的，mHealth JITAIs是自适应的，因为干预者 可以先验地指定决策规则，这些决策规则导致由某些 事件这些实验者指定的决策规则通常基于先前研究的结果， 特别是微随机试验，提供移动健康干预信息的顺序测试， 根据用户上下文确定消息的因果关系。然而，在此开发的JITAI 因为同样的决策规则同样适用于每个人， 而不考虑关于个人如何实际响应干预消息的信息。迅速 不断发展的机器学习方法，特别是强化学习（RL），使得提高 通过学习每个人独特的反应模式， 信息转化为后续的、个人特定的、自适应的决策规则。然而，少数行为 使用RL创建mHealth JITAIs减肥的干预者注意到， 这样做是因为RL的实现需要专业知识，并且可能是劳动密集型的。领域 需要一个用户友好的工具来减少这些障碍，以便RL方法被广泛采用。目的 1是开发Adapt，这是一种迭代集成实时数据的工具，应用RL算法， 进行微随机试验，以优化吉泰减肥决策规则。adapt会拉进来 实时数字健康数据，并使用行为模式和结果进行微随机试验， 达到最有效的干预信息，在正确的时间传递，以促进减肥， 每个参与者。目标2是进行一项为期12周的试点可行性研究， 减肥干预（NudgeRL）。NudgeRL将建立在团队现有的JITAI Nudge的基础上， 不包含RL。该样本将由20名超重或肥胖的成年人组成，其中至少50%是 黑人或拉丁裔虽然Adapt将被开发用于改善减肥干预措施，但其广泛使用将 在广泛的健康结果中产生更有效和更有效的JITAI，从而降低 癌症和其他疾病的负担，由于广泛的改善健康行为。