Collaborative Research: Combining Expert Judgments for Environmental Risk Analysis

与Clemen（0084383）和Gelman（0084368）的合作项目将开发和演示在环境风险分析中使用专家判断的改进方法。风险分析模型严重依赖于通过非正式或正式专家判断获得的参数估计。这种方法可能对两个问题很敏感：专家将他们的知识编码成概率分布的偏见（如过度自信偏见），以及专家之间判断的典型不确定依赖程度（例如，每个专家的判断结合了他对共同科学文献的阅读和他自己的经验和解释）。该项目将分析两种最先进的结合专家判断的方法的特性：由代尔夫特理工大学的罗杰·库克及其同事开发的“经典”方法和由杜克大学的罗伯特·克莱门及其同事开发的“copula”方法。它将比较数学方法，并通过评估它们在合成和实际专家判断数据集上的性能。此外，该项目将开发基于完全贝叶斯方法的改进方法，用于结合专家之间的过度自信和依赖，并将其与现有的两种方法进行比较。该项目将导致更好的方法和对环境风险分析中结合专家判断的替代方法的更好理解。由于物理或伦理原因，风险分析模型中的许多参数无法直接测量（例如，通常无法在远离污染源的地方追踪污染物，也无法对人体进行毒性研究）。因此，参数估计往往是基于专家判断。在大多数情况下，专家判断是非正式的，当模型构建者使用他们自己对参数值的“最佳猜测”估计时。在某些情况下（例如与核电有关的风险），判断（以概率分布的形式）是从专家小组中得出的。然而，目前还没有标准的方法来结合多个专家的判断，并且对替代方法的性质的理解有限。这项研究将导致更好地理解替代方法，并最终更好地利用专家判断。

This collaborative project with Clemen (0084383) and Gelman (0084368) will develop and demonstrate improved methods for using expert judgment in environmental risk analysis. Risk-analytic models rely heavily on parameter estimates obtained using either informal or formal expert judgments. Such methods can be sensitive to two problems: biases in experts coding their knowledge into probability distributions (such as overconfidence bias), and the typically uncertain degree of dependence in judgments between experts (e.g., each expert's judgment combines his reading of a common scientific literature and his own experience and interpretations). The project will analyze the properties of two state-of-the-art methods for combining expert judgments: the "classical" method developed by Roger Cooke and colleagues at Delft University of Technology and the "copula" method developed by Robert Clemen and colleagues at Duke University. It will compare the methods mathematically and by evaluating their performance on synthetic and actual expert judgment data sets. In addition, the project will develop improved methods for combining distributions based on fully Bayesian methods for incorporating overconfidence and dependence among experts, and compare them with the two existing methods.The project will lead to better methods and an improved understanding of alternative methods for combining expert judgment in environmental risk analysis. Many parameters in a risk-analysis model cannot be measured directly, for physical or ethical reasons (e.g., one typically cannot trace pollutants far from their source nor conduct toxicity studies on humans). As a result, parameter estimates are often based on expert judgment. In most cases, the expert judgment is applied informally, as when the model builders use their own "best guess" estimates of parameter values. In some cases (such as risks associated with nuclear power), judgments (in the form of probability distributions) are elicited from a panel of experts. However, there is at present no standard method for combining judgments from multiple experts, and limited understanding of the properties of alternative methods. This research will lead to better understanding of alternative methods and, ultimately, to better use of expert judgment.