A study of expert overconfidence
A study of expert overconfidence
复制标题
DOI：
10.1016/j.ress.2007.03.014
复制
发表时间：
2008-05-01
影响因子：
8.1
通讯作者：
Bier, Vicki M.
中科院分区：
文献类型：
--
作者：
Lin, Shi-Woei;Bier, Vicki M.
过度自信是专家判断中最常见（也是最严重）的问题之一。为了评估专家过度自信的程度，我们分析了一个由库克及其同事在德尔夫特技术大学和其他地方汇编的关于专家意见的大型数据集。该数据集包含大约5000个90%置信区间的不确定量，其真实值现在已知。我们的分析评估了数据集中过度自信的总体程度。在研究中，专家之间，以及研究中的问题之间，过度自信的程度存在显着差异。此外，重复（同一问题的多个实现）允许初步评估问题效果是否主要是由于问题的难度，或仅仅是随机噪声的不确定量的实现。这项分析的结果表明，许多明显的问题效应可能是由于噪声，而不是系统的差异，难以实现良好的校准不同的问题。研究结果支持专家的差异加权，因为研究中的专家校准存在显着差异。(c)2007爱思唯尔有限公司版权所有。
Overconfidence is one of the most common (and potentially severe) problems in expert judgment. To assess the extent of expert overconfidence, we analyzed a large data set on expert opinion compiled by Cooke and colleagues at the Technical University of Delft and elsewhere. This data set contains roughly five thousand 90% confidence intervals of uncertain quantities for which the true values are now known. Our analysis assesses the overall extent of overconfidence in the data set. Significant differences in the extent of overconfidence were found among studies, among experts, and among questions within a study. Moreover, replications (multiple realizations for the same question) allowed a preliminary assessment of whether the question effect is due largely to question difficulty, or merely to random noise in the realizations of the uncertain quantities. The results of this analysis, suggest that much of the apparent question effect may be due to noise rather than systematic differences in the difficulty of achieving good calibration for different questions. The results support the differential weighting of experts, since there are significant differences in expert calibration within studies. (c) 2007 Elsevier Ltd. All rights reserved.