Use of expert judgment in exposure assessment: part 2. Calibration of expert judgments about personal exposures to benzene.
Use of expert judgment in exposure assessment: part 2. Calibration of expert judgments about personal exposures to benzene.
复制标题
在暴露评估中使用专家判断：第 2 部分。关于个人苯暴露的专家判断的校准。
DOI：
10.1038/sj.jea.7500253
复制
发表时间：
2003
期刊：
影响因子：
--
通讯作者：
Evans,JohnS
中科院分区：
文献类型：
--
作者：
Walker,KatherineD;Catalano,Paul;Hammitt,JamesK;Evans,JohnS
监管机构最近对人类健康和环境风险进行概率分析的运动，更加关注其背后的可变性和不确定性估计的质量。特别值得关注的是如何表征不确定性（对未知事物的衡量标准），因为不确定性可以在分析监管控制需求或估计额外研究的经济价值时发挥重要作用。本文报告了作为国家人类暴露评估调查 (NHEXAS) 的一部分进行的第二阶段研究，旨在获取和校准暴露评估专家对美国 EPA 第五区不吸烟、非职业接触人群所经历的住宅环境、住宅室内和个人空气苯浓度不确定性的判断。关于每个苯类暴露的平均值和第 90 个百分位数的主观判断（即中位数、四分位距和 90% 置信区间）参与该研究的七位专家得出了苯的分布。专家判断的校准或质量是通过使用图形技术、二次评分规则以及惊喜指数和四分位数指数与 NHEXAS 第五区研究的实际测量值进行比较来评估的。两种定量评分方法的结果表明，综合考虑，专家的判断虽然总体上不够自信，但校准得相对较好。个别专家判断的校准似乎存在差异，凸显了依赖个别专家的潜在陷阱。令人惊讶的发现是，专家们对苯分布第 90 个百分位数的判断比他们对均值的预测得到了更好的校准；专家们往往对自己预测手段的能力过于自信。这篇论文也是最早证明考虑专家内部相关性对研究结果统计显着性的重要性的校准研究之一。当假设判断是独立的时，对意外指数和四分位数指数的分析发现了校准不良的证据（P<0.05）。然而，当考虑到研究中的专家内部相关性时，这些发现不再具有统计学意义。分析进一步发现，专家的判断得分高于对美国其他城市环境、室内和个人苯水平的早期研究得出的 V 区苯浓度估计值。这些结果表明，在以概率形式描述风险特征时，仔细引出专家判断的价值。需要进行额外的校准研究来证实和扩展这些发现。
The recent movement of regulatory agencies toward probabilistic analyses of human health and environmental risks has focused greater attention on the quality of the estimates of variability and uncertainty that underlie them. Of particular concern is how uncertainty—a measure of what is not known—is characterized, as uncertainty can play an influential role in analyses of the need for regulatory controls or in estimates of the economic value of additional research. This paper reports the second phase of a study, conducted as an element of the National Human Exposure Assessment Survey (NHEXAS), to obtain and calibrate exposure assessment experts judgments about uncertainty in residential ambient, residential indoor, and personal air benzene concentrations experienced by the nonsmoking, nonoccupationally exposed population in US EPA's Region V. Subjective judgments (ie, the median, interquartile range, and 90% confidence interval) about the means and 90th percentiles of each of the benzene distributions were elicited from the seven experts participating in the study. The calibration or quality of the experts' judgments was assessed by comparing them to the actual measurements from the NHEXAS Region V study using graphical techniques, a quadratic scoring rule, and surprise and interquartile indices. The results from both quantitative scoring methods suggested that, considered collectively, the experts' judgments were relatively well calibrated although on balance, underconfident. The calibration of individual expert judgments appeared variable, highlighting potential pitfalls in reliance on individual experts. In a surprising finding, the experts' judgments about the 90th percentiles of the benzene distributions were better calibrated than their predictions about the means; the experts tended to be overconfident in their ability to predict the means. This paper is also one of the first calibration studies to demonstrate the importance of taking into account intraexpert correlation on the statistical significance of the findings. When the judgments were assumed to be independent, analysis of the surprise and interquartile indices found evidence of poor calibration (P< 0.05). However, when the intraexpert correlation in the study was taken into account, these findings were no longer statistically significant. The analysis further found that the experts' judgments scored better than estimates of Region V benzene concentrations simply drawn from earlier studies of ambient, indoor and personal benzene levels in other US cities. These results suggest the value of careful elicitation of expert judgments in characterizing exposures in probabilistic form. Additional calibration studies need to be undertaken to corroborate and extend these findings.