Can a Single Variable Predict Early Dropout From Digital Health Interventions? Comparison of Predictive Models From Two Large Randomized Trials.
Can a Single Variable Predict Early Dropout From Digital Health Interventions? Comparison of Predictive Models From Two Large Randomized Trials.
复制标题
DOI：
10.2196/43629
复制
发表时间：
2023-01-20
影响因子：
7.4
通讯作者：
Vock, David M.
中科院分区：
文献类型：
--
作者：
Bricker, Jonathan;Miao, Zhen;Mull, Kristin;Santiago-Torres, Margarita;Vock, David M.
关键词：
一个可准确预测数字健康干预早期辍学的单一可推广指标有可能随时为干预目标和治疗增强提供信息，从而提高保留率和干预结果。我们最近发现了一种戒烟数字健康干预的早期退出者，特别是在干预的第一周登录并且此后几乎没有活动的用户。与使用iCanQuit戒烟应用程序较长时间的用户相比，这些用户使用该应用程序的戒烟率也大幅降低。本研究旨在探索使用标准统计方法的登录计数数据是否可以精确预测个体是否会成为iCanQuit早期退出者，同时使用其他统计方法和来自其他3种戒烟数字干预的随机试验数据验证该方法（合并随机N=4529）。使用标准逻辑回归模型预测接受iCanQuit戒烟干预应用程序、国家癌症研究所QuitGuide戒烟干预应用程序、WebQuit.org戒烟干预网站和Smokefree.gov戒烟干预网站的个人的早期脱落。主要预测因素是随机化后前7天内参与者每天登录的次数。曲线下面积（AUC）评估了逻辑回归模型的性能，并与决策树、支持向量机和神经网络模型进行了比较。我们还检查了13个基线变量，包括各种人口统计学（例如，种族和民族，性别和年龄）和吸烟特征（例如，使用电子烟和对无烟的信心）是否可以改善这一预测。iCanQuit、QuitGuide、www.example.com和www.example.com的仅使用前7天登录计数变量的各逻辑回归模型的AUC分别为0.94（95% CI 0.90-0.97）、0.88（95% CI 0.83-0.93）、0.85（95% CI 0.80-0.88）和0.60（95% CI 0.90-0.97）WebQuit.org。用更复杂的决策树、支持向量机或神经网络模型替换逻辑回归模型并没有显著增加AUC，也没有包括额外的基线变量作为预测因子。灵敏度和特异性通常良好，iCanQuit的灵敏度和特异性非常好（即，在0.5分类阈值下分别为0.91和0.85）。仅使用前7天的登录计数数据的逻辑回归模型通常能够很好地预测早期脱落。这些模型在使用简单、自动化和容易获得的登录计数数据时表现良好，而包括自我报告的基线变量并没有改善预测。研究结果将为早期识别数字健康干预措施中有早期辍学风险的人提供信息，目的是通过为他们提供增强治疗来进一步干预，以增加他们的保留率，并最终增加他们的干预结果。
A single generalizable metric that accurately predicts early dropout from digital health interventions has the potential to readily inform intervention targets and treatment augmentations that could boost retention and intervention outcomes. We recently identified a type of early dropout from digital health interventions for smoking cessation, specifically, users who logged in during the first week of the intervention and had little to no activity thereafter. These users also had a substantially lower smoking cessation rate with our iCanQuit smoking cessation app compared with users who used the app for longer periods. This study aimed to explore whether log-in count data, using standard statistical methods, can precisely predict whether an individual will become an iCanQuit early dropout while validating the approach using other statistical methods and randomized trial data from 3 other digital interventions for smoking cessation (combined randomized N=4529). Standard logistic regression models were used to predict early dropouts for individuals receiving the iCanQuit smoking cessation intervention app, the National Cancer Institute QuitGuide smoking cessation intervention app, the WebQuit.org smoking cessation intervention website, and the Smokefree.gov smoking cessation intervention website. The main predictors were the number of times a participant logged in per day during the first 7 days following randomization. The area under the curve (AUC) assessed the performance of the logistic regression models, which were compared with decision trees, support vector machine, and neural network models. We also examined whether 13 baseline variables that included a variety of demographics (eg, race and ethnicity, gender, and age) and smoking characteristics (eg, use of e-cigarettes and confidence in being smoke free) might improve this prediction. The AUC for each logistic regression model using only the first 7 days of log-in count variables was 0.94 (95% CI 0.90-0.97) for iCanQuit, 0.88 (95% CI 0.83-0.93) for QuitGuide, 0.85 (95% CI 0.80-0.88) for WebQuit.org, and 0.60 (95% CI 0.54-0.66) for Smokefree.gov. Replacing logistic regression models with more complex decision trees, support vector machines, or neural network models did not significantly increase the AUC, nor did including additional baseline variables as predictors. The sensitivity and specificity were generally good, and they were excellent for iCanQuit (ie, 0.91 and 0.85, respectively, at the 0.5 classification threshold). Logistic regression models using only the first 7 days of log-in count data were generally good at predicting early dropouts. These models performed well when using simple, automated, and readily available log-in count data, whereas including self-reported baseline variables did not improve the prediction. The results will inform the early identification of people at risk of early dropout from digital health interventions with the goal of intervening further by providing them with augmented treatments to increase their retention and, ultimately, their intervention outcomes.
登录
查看更多内容
影响因子：
2.9
作者：
Bothwell LE;Avorn J;Khan NF;Kesselheim AS
通讯作者：
Kesselheim AS
影响因子：
7.4
作者：
Flores Mateo G;Granado-Font E;Ferré-Grau C;Montaña-Carreras X
通讯作者：
Montaña-Carreras X
影响因子：
5
作者：
Grutzmacher, Stephanie K.;Munger, Ashley L.;Lachenmayr, Lisa
通讯作者：
Lachenmayr, Lisa
影响因子：
7.4
作者：
Bremer V;Chow PI;Funk B;Thorndike FP;Ritterband LM
通讯作者：
Ritterband LM
影响因子：
5.2
作者：
Gentili, Andrea;Failla, Giovanna;Melnyk, Andriy;Puleo, Valeria;Tanna, Gian Luca Di;Ricciardi, Walter;Cascini, Fidelia
通讯作者：
Cascini, Fidelia