Algorithmic Methods for Combining Expert Judgments

结合专家判断的算法方法

• 批准号: 0720858
• 负责人: James Hammitt
• 金额: $ 19.1万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-08-15 至 2010-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0720858&HistoricalAwards=false
• 关键词: Algorithmic; Methods; Combining; Expert; Judgments

Decisions about environmental protection, homeland security, and other critical issues are informed by mathematical models used to estimate the risk and consequences of alternative actions. Many of the parameters in these models cannot be measured directly and so models rely heavily on expert judgment. In recent years, formal structured methods for eliciting judgments from a panel of subject-matter experts have been developed and applied to important public issues (e.g., nuclear power safety, air pollution). In structured expert judgment, each expert reports his or her beliefs about the value of a parameter as a subjective probability distribution. For decision making, it is useful to combine the experts' judgments into a single probability distribution. This research investigates the performance of alternative combination rules, including a simple average, a weighted average with weights based on the experts' performance on variables for which the true values become known, and a method that updates the decision maker's distribution in accordance with Bayes' Rule. The research compares the performance of these methods by using synthetic expert judgments in which the number of experts and the qualities of their individual judgments (e.g., precision, calibration) are known. The project will also use real expert-judgment data obtained for environmental risk assessment to see how sensitive results are to alternative combination methods and analytic choices. The work should contribute to better decision making through improved understanding of methods for using experts' judgments, including questions of how many and how diverse a set of experts to involve and how to synthesize their responses.

有关环境保护、国土安全和其他关键问题的决策都是通过数学模型来评估替代行动的风险和后果。这些模型中的许多参数无法直接测量，因此模型严重依赖于专家的判断。近年来，用于从主题专家小组中引出判断的正式结构化方法已经被开发并应用于重要的公共问题（例如，核电安全、空气污染）。在结构化专家判断中，每个专家报告他或她对参数值的信念作为主观概率分布。对于决策制定，将专家的判断联合收割机组合成单个概率分布是有用的。本研究探讨了替代组合规则的性能，包括一个简单的平均值，加权平均值的权重的基础上专家的表现变量的真实值成为已知的，和一种方法，更新决策者的分布按照贝叶斯规则。该研究通过使用综合专家判断来比较这些方法的性能，其中专家的数量和他们个人判断的质量（例如，精度、校准）是已知的。该项目还将使用环境风险评估所获得的真实的专家判断数据，以了解结果对替代组合方法和分析选择的敏感程度。这项工作应有助于通过更好地了解使用专家判断的方法来更好地作出决策，包括涉及多少专家和专家的多样性以及如何综合他们的反应等问题。