ASSESSING PREDICTIVE ACCURACY - HOW TO COMPARE BRIER SCORES
ASSESSING PREDICTIVE ACCURACY - HOW TO COMPARE BRIER SCORES
复制标题
DOI：
10.1016/0895-4356(91)90146-z
复制
发表时间：
1991-01-01
影响因子：
7.2
通讯作者：
HICKAM, DH
中科院分区：
文献类型：
--
作者：
REDELMEIER, DA;BLOCH, DA;HICKAM, DH
一些研究者已经使用Brier指数来衡量一系列医学判断的预测准确性；对同一患者进行评估的不同评分者的Brier分数提供了相对准确性的衡量标准。然而，由于缺乏区分两个Brier分数的统计测试，这种比较可能难以解释。为了证明解决这一问题的方法，我们分析了五名医学生的判断，他们每个人都独立地评估了25名复发性胸痛患者。使用该方法，我们确定有2名学生给出的判断与实际观察结果不一致（p < 0.05）；在剩下的三名学生中，我们发现两名学生之间有显著差异（p < 0.05）。这些结果不同于接受者工作特征曲线面积分析，这是另一种用于评估预测准确性的技术。我们建议所提出的方法可以为研究者提供一个有用的工具，使用Brier指数来比较临床医生使用概率判断表达不确定性的程度。
Several investigators have used the Brier index to measure the predictive accuracy of a set of medical judgments; the Brier scores of different raters who have evaluated the same patients provides a measure of relative accuracy. However, such comparisons may be difficult to interpret because of the lack of a statistical test for differentiating between two Brier scores. To demonstrate a method for addressing this issue we analyzed the judgments of five medical students, each of whom independently evaluated the same 25 patients with recurrent chest pain. Using the method we determined that two of the students gave judgments that were incompatible with the actual observed outcomes (p < 0.05); of the three remaining students we detected a significant difference between two (p < 0.05). These results differed from receiver operating characteristic curve area analysis, another technique used to evaluate predictive accuracy. We suggest that the proposed method can provide a useful tool for investigators using the Brier index to compare how well clinicians express uncertainty using probability judgments.