Prospective validation study of an epilepsy seizure risk system for outpatient evaluation
Prospective validation study of an epilepsy seizure risk system for outpatient evaluation
复制标题
DOI：
10.1111/epi.16397
复制
发表时间：
2019-12-02
期刊：
影响因子：
5.6
通讯作者：
Stern, John M.
中科院分区：
文献类型：
--
作者：
Chiang, Sharon;Goldenholz, Daniel M.;Stern, John M.
目的我们使用癫痫发作计数数据对自动贝叶斯机器学习算法（癫痫发作评估工具[EpiSAT]）进行了门诊癫痫发作风险评估的临床测试，并对专业癫痫临床专家进行了性能验证。方法我们对美国三个三级转诊癫痫中心的24名专业临床专家进行了一项前瞻性纵向研究。使用来自4份合成癫痫发作日记（癫痫发作风险已知）的120次癫痫发作和来自4份真实的癫痫发作日记（癫痫发作风险未知）的120次癫痫发作，评价了EpiSAT正确识别癫痫发作风险变化（改善、恶化或无变化）的准确性、评分者间可靠性和评分者内可靠性。评价EpiSAT与临床医生之间观察到的一致性的比例，以评估EpiSAT与癫痫专家临床决策模式的兼容性。结果EpiSAT与临床医生评估癫痫发作风险的一致率为75.4%。癫痫提供者正确评估癫痫发作风险的平均准确率为74.7%。EpiSAT准确识别了87.5%的癫痫发作日记条目中的癫痫发作风险，对应于17.4%的显著改善（P = .002）。临床医生在4- 12周的评估期内表现出低至中等的癫痫风险评估者间可靠性（Krippendorff α = 0.46）和良好的评估者间可靠性（Scott pi = 0.89）。这些结果验证了EpiSAT产生癫痫发作风险的客观临床建议的能力，这些建议遵循与专业癫痫提供者相似的决策模式，但具有更高的准确性和可重复性。该算法可作为临床决策支持系统，用于癫痫临床实践中临床发作频率的定量分析。
Objective We conducted clinical testing of an automated Bayesian machine learning algorithm (Epilepsy Seizure Assessment Tool [EpiSAT]) for outpatient seizure risk assessment using seizure counting data, and validated performance against specialized epilepsy clinician experts. Methods We conducted a prospective longitudinal study of EpiSAT performance against 24 specialized clinician experts at three tertiary referral epilepsy centers in the United States. Accuracy, interrater reliability, and intra-rater reliability of EpiSAT for correctly identifying changes in seizure risk (improvements, worsening, or no change) were evaluated using 120 seizures from four synthetic seizure diaries (seizure risk known) and 120 seizures from four real seizure diaries (seizure risk unknown). The proportion of observed agreement between EpiSAT and clinicians was evaluated to assess compatibility of EpiSAT with clinical decision patterns by epilepsy experts. Results EpiSAT exhibited substantial observed agreement (75.4%) with clinicians for assessing seizure risk. The mean accuracy of epilepsy providers for correctly assessing seizure risk was 74.7%. EpiSAT accurately identified seizure risk in 87.5% of seizure diary entries, corresponding to a significant improvement of 17.4% (P = .002). Clinicians exhibited low-to-moderate interrater reliability for seizure risk assessment (Krippendorff's alpha = 0.46) with good intrarater reliability across a 4- to 12-week evaluation period (Scott's pi = 0.89). Significance These results validate the ability of EpiSAT to yield objective clinical recommendations on seizure risk which follow decision patterns similar to those from specialized epilepsy providers, but with improved accuracy and reproducibility. This algorithm may serve as a useful clinical decision support system for quantitative analysis of clinical seizure frequency in clinical epilepsy practice.