Validating vignette and conjoint survey experiments against real-world behavior
Validating vignette and conjoint survey experiments against real-world behavior
复制标题
DOI：
10.1073/pnas.1416587112
复制
发表时间：
2015-02-24
影响因子：
11.1
通讯作者：
Yamamoto, Teppei
中科院分区：
文献类型：
--
作者：
Hainmueller, Jens;Hangartner, Dominik;Yamamoto, Teppei
调查实验，如小插曲和联合分析，被广泛用于社会科学，以引出陈述的偏好和研究人类如何做出多维选择。然而，有一个缺乏的研究，这些方法的外部有效性，检查是否的决定因素，解释假设的选择调查受访者匹配的决定因素，解释什么样的主题实际上做的时候，在现实世界中的情况下，做出类似的选择。本研究比较了结果，从联合和小插曲分析，移民属性产生的支持入籍与密切对应的行为数据，从一个自然的实验在瑞士，在那里一些城市使用公民投票来决定外国居民的公民身份申请。使用来自同一人群的代表性样本和选民在每次公投之前收到的申请人特征的官方描述作为行为基准，我们发现，从调查实验中估计的申请人属性的影响在恢复行为基准中相同属性的影响方面表现得非常好。我们还发现不同设计的相对性能的重要差异。总体而言，配对联合设计，即受访者并排评估两名移民，最接近行为基准;平均而言，其估计值与行为基准的影响相差不到2%。
Survey experiments, like vignette and conjoint analyses, are widely used in the social sciences to elicit stated preferences and study how humans make multidimensional choices. However, there is a paucity of research on the external validity of these methods that examines whether the determinants that explain hypothetical choices made by survey respondents match the determinants that explain what subjects actually do when making similar choices in real-world situations. This study compares results from conjoint and vignette analyses on which immigrant attributes generate support for naturalization with closely corresponding behavioral data from a natural experiment in Switzerland, where some municipalities used referendums to decide on the citizenship applications of foreign residents. Using a representative sample from the same population and the official descriptions of applicant characteristics that voters received before each referendum as a behavioral benchmark, we find that the effects of the applicant attributes estimated from the survey experiments perform remarkably well in recovering the effects of the same attributes in the behavioral benchmark. We also find important differences in the relative performances of the different designs. Overall, the paired conjoint design, where respondents evaluate two immigrants side by side, comes closest to the behavioral benchmark; on average, its estimates are within 2% percentage points of the effects in the behavioral benchmark.