When can history be our guide? The pitfalls of counterfactual inference
When can history be our guide? The pitfalls of counterfactual inference
复制标题
DOI：
10.1111/j.1468-2478.2007.00445.x
复制
发表时间：
2007-03-01
影响因子：
2.6
通讯作者：
Zeng, Langche
中科院分区：
文献类型：
--
作者：
King, Gary;Zeng, Langche
关于反事实的推论对于预测、回答“如果”问题和估计因果效应是必不可少的。然而，当提出的反事实与手头的数据相差太远时，从精心指定的统计分析中得出的结论就变成了基于猜测和方便但站不住脚的模型假设，而不是经验证据。不幸的是，标准的统计方法假设模型的准确性，而不是揭示模型依赖的程度，所以这个问题很难发现。我们开发了易于应用的方法来评估反事实，不需要对指定类别的模型进行敏感性测试。如果一个分析没有通过我们提供的测试，那么我们知道，实质性的结果至少对一些不基于经验证据的模型选择是敏感的。我们使用这些方法来评估有关一个国家民主程度变化的影响（对任何因变量）的广泛学术文献，以及对联合国建设和平工作影响的单独分析。我们发现有证据表明，许多学者无意中得出的结论更多地基于建模假设，而不是数据中的证据。对于一些研究问题，历史包含的信息不足以成为我们的指南。本文附带的自由软件实现了我们所有的建议。
Inferences about counterfactuals are essential for prediction, answering "what if" questions, and estimating causal effects. However, when the counterfactuals posed are too far from the data at hand, conclusions drawn from well-specified statistical analyses become based on speculation and convenient but indefensible model assumptions rather than empirical evidence. Unfortunately, standard statistical approaches assume the veracity of the model rather than revealing the degree of model-dependence, so this problem can be hard to detect. We develop easy-to-apply methods to evaluate counterfactuals that do not require sensitivity testing over specified classes of models. If an analysis fails the tests we offer, then we know that substantive results are sensitive to at least some modeling choices that are not based on empirical evidence. We use these methods to evaluate the extensive scholarly literatures on the effects of changes in the degree of democracy in a country (on any dependent variable) and separate analyses of the effects of UN peacebuilding efforts. We find evidence that many scholars are inadvertently drawing conclusions based more on modeling hypotheses than on evidence in the data. For some research questions, history contains insufficient information to be our guide. Free software that accompanies this paper implements all our suggestions.