Development and validation of a novel model for characterizing migraine outcomes within real-world data.
Development and validation of a novel model for characterizing migraine outcomes within real-world data.
复制标题
DOI：
10.1186/s10194-022-01493-x
复制
发表时间：
2022-09-21
期刊：
影响因子：
--
通讯作者：
中科院分区：
文献类型：
--
作者：
在偏头痛或抑郁症等具有“软”结果（即医疗状况或其管理的主观方面）的疾病领域，由于数据的收集和记录方式，从电子健康记录 (EHR) 和其他常规收集的数据中提取和验证真实世界证据 (RWE) 可能具有挑战性。在这项研究中，我们的目的是定义和验证一个可扩展的框架模型，通过在 EHR 数据中使用人工智能 (AI) 算法来衡量偏头痛治疗和预防的结果。头痛专家根据常规收集的临床数据定义了描述性特征。对数据元素进行加权，定义一个 10 分制的量表，包括头痛严重程度（1-7 分）和相关特征（0-3 分）。确定了测试数据集，并由经过培训的注释者手动生成了参考标准。自动化（即人工智能）用于从患者就诊的非结构化数据中提取特征，并与参考标准进行比较。自动评分和人工注释者之间 70% 接近一致（1 分以内）的阈值被认为是足够的提取精度。还评估了人工智能识别用于构建结果模型的特征的准确性，成功被定义为在识别遭遇时达到 80% 的 F1 分数（即精确度和召回率的加权调和平均值）。使用 2,006 次遭遇的数据，识别出 11 个特征并将其包含在模型中；对于应用于非结构化数据的人工智能，自动提取的平均 F1 分数为 92.0%。与手动提取分数相比，结果模型在描述偏头痛状态方面具有出色的准确性，准确匹配率为 77.2%，接近匹配（1 分以内）为 82.2%，远高于研究前设置的 70% 匹配阈值。我们的研究结果表明，利用技术支持的模型可以使用通常在现实世界临床环境中捕获的数据元素来验证偏头痛进展等软结果的确定，从而为基于 EHR 的可信临床研究提供可扩展的方法。
In disease areas with ‘soft’ outcomes (i.e., the subjective aspects of a medical condition or its management) such as migraine or depression, extraction and validation of real-world evidence (RWE) from electronic health records (EHRs) and other routinely collected data can be challenging due to how the data are collected and recorded. In this study, we aimed to define and validate a scalable framework model to measure outcomes of migraine treatment and prevention by use of artificial intelligence (AI) algorithms within EHR data. Headache specialists defined descriptive features based on routinely collected clinical data. Data elements were weighted to define a 10-point scale encompassing headache severity (1–7 points) and associated features (0–3 points). A test data set was identified, and a reference standard was manually produced by trained annotators. Automation (i.e., AI) was used to extract features from the unstructured data of patient encounters and compared to the reference standard. A threshold of 70% close agreement (within 1 point) between the automated score and the human annotator was considered to be a sufficient extraction accuracy. The accuracy of AI in identifying features used to construct the outcome model was also evaluated and success was defined as achieving an F1 score (i.e., the weighted harmonic mean of the precision and recall) of 80% in identifying encounters. Using data from 2,006 encounters, 11 features were identified and included in the model; the average F1 scores for automated extraction were 92.0% for AI applied to unstructured data. The outcome model had excellent accuracy in characterizing migraine status with an exact match for 77.2% of encounters and a close match (within 1 point) for 82.2%, compared with manual extraction scores—well above the 70% match threshold set prior to the study. Our findings indicate the feasibility of technology-enabled models for validated determination of soft outcomes such as migraine progression using the data elements typically captured in the real-world clinical setting, providing a scalable approach to credible EHR-based clinical studies.
登录
查看更多内容
影响因子：
3.8
作者：
Blonde L;Khunti K;Harris SB;Meizinger C;Skolnik NS
通讯作者：
Skolnik NS
DOI：
10.1016/j.ijmedinf.2015.06.011
发表时间：
2015-10-01
影响因子：
4.9
作者：
Wright, Adam;McCoy, Allison B.;Sittig, Dean F.
通讯作者：
Sittig, Dean F.
影响因子：
39.2
作者：
Luce, Bryan R.;Kramer, Judith M.;Schwartz, J. Sanford
通讯作者：
Schwartz, J. Sanford
影响因子：
13.8
作者：
Kroth, Philip J.;Morioka-Douglas, Nancy;Linzer, Mark
通讯作者：
Linzer, Mark
影响因子：
2.7
作者：
Camm AJ;Fox KAA
通讯作者：
Fox KAA