Generating and Evaluating Interval Estimates

生成和评估区间估计

• 批准号: 0551225
• 负责人: Craig McKenzie
• 金额: $ 25万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-05-01 至 2010-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0551225&HistoricalAwards=false
• 关键词: Generating; Evaluating; Interval; Estimates

This research examines expressions of uncertainty in the form of interval estimates. For example, a realtor might estimate the market value of a home to be between $500,000 and $525,000, and more (less) uncertainty in the value would presumably result in a wider (narrower) interval. Previous research has shown that people tend to be overconfident when reporting confidence intervals, or CIs (e.g., "I'm 90% confident that the population of San Diego is between 1 million and 1.5 million"). That is, X% CIs tend to be much too narrow and contain the true value much less than X% of the time. This can matter a great deal. For example, a leading U.S. manufacturer elicited a projected range of sales from its marketing staff in order to plan the production capacity of a new factory. The range turned out to be too narrow and the new factory was incapable of meeting the unexpected demand.We examine both how people produce interval estimates and the closely related question of how people evaluate them. In evaluation tasks, evaluators are presented with interval estimates reported by multiple producers to the same question, told the true value, and asked which estimate is best. People tend to trade off an interval's accuracy (how close its midpoint is to the true value) and its informativeness (how narrow it is). For example, evaluators sometimes find intervals that do not contain the true value to be superior to those that do -- but only when the former are narrow, or highly informative. We propose a formal model to account for such behavior. Assuming that producers' subjective probability distributions are normal, our model calculates for each interval the subjective probability density at the true value. The interval with the highest density at the true value is judged superior. The only free parameter in the model is how much confidence the producer is assumed to have in the reported interval (e.g., whether the evaluator treats it as a 50% CI or a 90% CI). Preliminary results indicate that the new model outperforms competing models. Because we view interval production and evaluation as two sides of the same coin -- how intervals are evaluated may shape how they are produced -- the results from the proposed evaluation experiments will guide research on production. In addition, we are conducting a complementary line of research on production. Our starting point is the fact that producers are largely insensitive to explicit probabilities manipulated between subjects, but they are sensitive to the probabilities manipulated within subjects. Our experiments will reveal subjects' expectations about their hit rate (e.g., do subjects even expect their 90% CIs to contain the true value more often than their 50% CIs?), and the nature of their within-subject sensitivity to explicit probabilities (e.g., are producers responding to the explicit probabilities, or are they merely widening and narrowing their intervals in the direction implied by the numbers?). These experiments will answer basic questions about interval production that have both applied and theoretical implications.

本研究以区间估计的形式检验不确定性的表达。例如，房地产经纪人可能估计房屋的市场价值在50万美元到52.5万美元之间，价值的不确定性越大（越少），可能导致区间越宽（越窄）。先前的研究表明，人们在报告置信区间或ci时往往过于自信（例如，“我有90%的信心认为圣地亚哥的人口在100万到150万之间”）。也就是说，X%的ci往往过于狭窄，包含的真实值远远少于X%的时间。这很重要。例如，一家领先的美国制造商为了规划新工厂的生产能力，从营销人员那里得到了预计的销售范围。这个范围太窄了，新工厂无法满足意外的需求。我们研究了人们如何产生区间估计，以及人们如何评估它们的密切相关的问题。在评估任务中，评估者会看到由多个生产者对同一个问题报告的间隔估计，并被告知真实值，并被问及哪个估计是最好的。人们倾向于权衡区间的准确性（它的中点与真实值的接近程度）和信息量（它有多窄）。例如，评估者有时会发现不包含真实值的区间优于包含真实值的区间——但只有当前者很窄或信息量很大时才会如此。我们提出一个正式的模型来解释这种行为。假设生产者的主观概率分布为正态分布，我们的模型计算每个区间真实值下的主观概率密度。在真实值处密度最高的区间被认为是优越的。模型中唯一的自由参数是假设生产者在报告的区间内有多大的信心（例如，评估者是否将其视为50% CI或90% CI）。初步结果表明，新模型优于竞争模型。因为我们将储层开采和评估视为同一枚硬币的两面——储层的评估方式可能会影响储层的开采方式——所提出的评估实验的结果将指导生产研究。此外，我们正在进行生产方面的补充研究。我们的出发点是，生产者对主体之间操纵的显性概率基本上不敏感，但他们对主体内部操纵的概率很敏感。我们的实验将揭示受试者对命中率的期望（例如，受试者是否期望90%的ci比50%的ci更能包含真实值？），以及他们对显式概率的内在敏感性（例如，制作人是否对显式概率做出反应，或者他们只是在数字暗示的方向上扩大或缩小间隔？）。这些实验将回答有关间歇生产的基本问题，这些问题具有应用和理论意义。