Training and Interpreting Machine Learning Algorithms to Evaluate Fall Risk After Emergency Department Visits
Training and Interpreting Machine Learning Algorithms to Evaluate Fall Risk After Emergency Department Visits
复制标题
DOI：
10.1097/mlr.0000000000001140
复制
发表时间：
2019-07-01
期刊：
影响因子：
3
通讯作者：
Shah, Manish N.
中科院分区：
文献类型：
--
作者：
Patterson, Brian W.;Engstrom, Collin J.;Shah, Manish N.
背景：机器学习越来越多地用于医疗保健中的风险分层。如果不能将准确的预测模型转化为有效的干预措施，则无法改善结果。在这里，我们研究了自动风险分层和转诊干预的潜在效用，以筛选老年人在急诊科（艾德）就诊后的跌倒风险。目的：本研究评估了使用电子健康记录数据创建风险分层算法的几种机器学习方法，并根据测试数据中的算法性能估计了最终干预的效果。方法：回顾性收集艾德出院时可用的数据，并将其分为训练和测试数据集。开发了算法来预测艾德指数访视后6个月内跌倒的回访结果。模型包括随机森林，AdaBoost和基于回归的方法。我们通过受试者工作特征（ROC）曲线下面积（也称为曲线下面积（AUC））和预测的临床影响，估计每周需要治疗的人数（NNT）和转诊跌倒风险干预来评估模型。结果：随机森林模型的AUC为0.78，基于回归的模型的性能略低。当通过AUC进行评估时，具有相似性能的算法在放入临床背景中并在现实场景中执行估计NNT的定义任务时会有所不同。结论：将我们的分析结果转化为转诊数量和NNT之间的潜在权衡的能力为决策者提供了在实施前设想拟议干预措施的影响的能力。
Background: Machine learning is increasingly used for risk stratification in health care. Achieving accurate predictive models do not improve outcomes if they cannot be translated into efficacious intervention. Here we examine the potential utility of automated risk stratification and referral intervention to screen older adults for fall risk after emergency department (ED) visits. Objective: This study evaluated several machine learning methodologies for the creation of a risk stratification algorithm using electronic health record data and estimated the effects of a resultant intervention based on algorithm performance in test data. Methods: Data available at the time of ED discharge were retrospectively collected and separated into training and test datasets. Algorithms were developed to predict the outcome of a return visit for fall within 6 months of an ED index visit. Models included random forests, AdaBoost, and regression-based methods. We evaluated models both by the area under the receiver operating characteristic (ROC) curve, also referred to as area under the curve (AUC), and by projected clinical impact, estimating number needed to treat (NNT) and referrals per week for a fall risk intervention. Results: The random forest model achieved an AUC of 0.78, with slightly lower performance in regression-based models. Algorithms with similar performance, when evaluated by AUC, differed when placed into a clinical context with the defined task of estimated NNT in a real-world scenario. Conclusion: The ability to translate the results of our analysis to the potential tradeoff between referral numbers and NNT offers decisionmakers the ability to envision the effects of a proposed intervention before implementation.