SBIR Phase II: Confusion to Precision: Tools for Assessing Expert Uncertainty

小行星9530293 定量模型几乎都存在不确定性，而且大多数模型必须依靠专家意见来估计关键的不确定投入量，这些模型越来越广泛地用于支持公共和私营部门的关键决策。 这个小型企业创新研究第二阶段项目来自Lumina决策系统公司。描述了寻求开发和评估一种创新的决策支持系统软件工具的研究，以帮助专家以概率分布的形式表达他们的不确定知识。 有一个越来越多的认识的建设者和用户的需要这样一个工具的定量模型的一部分，但迄今为止的使用一直受到限制，缺乏可用的工具，既易于使用，旨在克服困难的人表达的不确定性，在有意义的和连贯的方式。 在第一阶段，开发和实验建立了两种创新方法的实用性：（1）动画密度函数和（2）口头概率短语，如可能或很少。 在第二阶段，将完善概率评估的原型工具，为这两种和其他六种评估方法提供相关的校准和去偏技术。这些方法将通过实验进行评估，以衡量学习和使用的容易程度、所需的努力、结果的置信度以及所得分布的可靠性和校准。 这些结果将用于指导用户选择最具成本效益的方法，以满足其需求和技能。 评估多元分布的方法和概率评估的多媒体课件也将被集成到软件包中。 在不确定性很重要的领域，包括金融、石油和天然气勘探、环境风险、医疗诊断和投标决策等领域，专家判断以主观概率分布的形式表达正逐渐被接受用于风险分析。 这一项目将提供有效手段，使人们相信结果的可能性，从而有助于加强现有的在各种情况下量化预期成本和效益的能力。

9530293 Sonnenblick Quantitative models, virtually all of which are subject to uncertainties and most of which must rely on expert opinion for estimating key uncertain input quantities, are increasingly widely used to support the making of key decisions in public and private sectors. This Small Business Innovation Research Phase II project from Lumina Decision Systems, Inc. describes research seeking to develop and evaluate an innovative decision support system software tool to assist experts in expressing their uncertain knowledge in the form of probability distributions. There is a growing recognition on the part of builders and users of quantitative models of the need for a such a tool, but heretofore usage has been constrained by the lack of availability of a tool that is both easy to use and designed to overcome the difficulties people have expressing uncertainty in meaningful and coherent ways. In Phase I, the practicality of two innovative methods was developed and experimentally established: (1) animated density functions and (2) verbal probability phrases, such as probable or very seldom. In Phase II, a prototype tool for probability assessment will be refined, providing these two and six other assessment methods with associated calibration and debiasing techniques...These methods will be evaluated experimentally to measure ease of learning and use, effort required, confidence in results, and reliability and calibration of the resulting distributions. These results will be used to guide users in choosing the most cost-effective methods to suit their needs and skills. Methods to assess multivariate distributions and multimedia courseware on probability assessment will also be integrated into the package. The expression of expert judgment in the form of subjective probability distributions is becoming accepted for risk analysis in areas where uncertainty is critical, including finance, oil and gas exploration, risks to the environment, medical diagnosis, and bidding decisions . This project will contribute effective means of eliciting beliefs regarding the likelihood of outcomes, and thus it will serve to enhance existing capability of quantifying expected costs and benefits in a variety of contexts.