Measuring expertise in identifying interictal epileptiform discharges.
Measuring expertise in identifying interictal epileptiform discharges.
复制标题
DOI：
10.1684/epd.2021.1409
复制
发表时间：
2022-06-01
影响因子：
2.3
通讯作者：
Westover, Michael Brandon
中科院分区：
文献类型：
--
作者：
Harid, Nitish M;Jing, Jin;Hogan, Jacob;Nascimento, Fabio A;Ouyang, An;Zheng, Wei-Long;Ge, Wendong;Zafar, Sahar F;Kim, Jennifer A;Alice, D Lam;Herlopian, Aline;Maus, Douglas;Karakis, Ioannis;Ng, Marcus;Hong, Shenda;Yu, Zhu;Kaplan, Peter W;Cash, Sydney;Shafi, Mouhsin;Martz, Gabriel;Halford, Jonathan J;Westover, Michael Brandon
脑电图上的发作间期癫痫样放电是诊断癫痫的重要组成部分。然而，脑电图由接受过或未接受过专业培训的读者来解释，并且没有公认的方法来评估解释技能。我们的目标是开发一种测试来量化简易爆炸装置识别技能。从 EEG 中选出了总共 13,262 个候选 IED，并由 8 名接受过奖学金培训的评审员进行评分，以建立黄金标准。开发了在线测试来评估不同训练水平的读者区分候选波形的能力。计算每个读数器的灵敏度、假阳性率和校准。我们开发了一个简单的数学模型来估计每个读者识别简易爆炸装置的技能和阈值，并为每个读者开发接收器操作特性曲线。我们调查了以可接受的精度测量技能水平所需的简易爆炸装置的数量。二十九名评估者完成了测试；九名专家、七名经验丰富的非专家和十三名新手。专家、经验丰富的非专家和新手的中位校准误差分别为 -0.056、0.012、0.046；中位敏感性分别为 0.800、0.811、0.715；假阳性率中位数分别为 0.177、0.272、0.396。衡量这些分数所需的测试题数量为 549。我们的分析发现，与经验丰富的非专家和专家相比，新手的噪音水平（不确定性）更高。使用计算出的噪声和阈值水平，创建了接收者操作曲线，显示从新手（0.735）到经验丰富的非专家（0.852）和专家（0.891），曲线下的中值面积不断增加。可以根据识别简易爆炸装置的能力来区分专家和非专家读者。这种类型的评估还可用于识别和纠正识别简易爆炸装置的阈值差异。
Interictal epileptiform discharges on EEG are integral to diagnosing epilepsy. However, EEGs are interpreted by readers with and without specialty training, and there is no accepted method to assess skill in interpretation. We aimed to develop a test to quantify IED recognition skills. A total of 13,262 candidate IEDs were selected from EEGs and scored by eight fellowship-trained reviewers to establish a gold standard. An online test was developed to assess how well readers with different training levels could distinguish candidate waveforms. Sensitivity, false positive rate and calibration were calculated for each reader. A simple mathematical model was developed to estimate each reader’s skill and threshold in identifying an IED, and to develop receiver operating characteristics curves for each reader. We investigated the number of IEDs needed to measure skill level with acceptable precision. Twenty-nine raters completed the test; nine experts, seven experienced non-experts and thirteen novices. Median calibration errors for experts, experienced non-experts and novices were −0.056, 0.012, 0.046; median sensitivities were 0.800, 0.811, 0.715; and median false positive rates were 0.177, 0.272, 0.396, respectively. The number of test questions needed to measure those scores was 549. Our analysis identified that novices had a higher noise level (uncertainty) compared to experienced non-experts and experts. Using calculated noise and threshold levels, receiver operating curves were created, showing increasing median area under the curve from novices (0.735), to experienced non-experts (0.852) and experts (0.891). Expert and non-expert readers can be distinguished based on ability to identify IEDs. This type of assessment could also be used to identify and correct differences in thresholds in identifying IEDs.