Can Internet Surveys Be Trusted?

基于概率的互联网调查小组有可能以低成本和提供复杂信息处理的能力，从代表一般人口的样本中提供调查答复。尽管使用随机数字拨号来招募小组成员，但鉴于小组成员招募和保留的多个阶段，这一技术引发了对调查答复代表性的担忧。对于管理模式，如在电脑上进行调查，如何影响调查反应，人们也知之甚少。本研究项目将寻求以下问题的答案：(1)管理模式和抽样策略引入了哪些无反应偏差？具体来说，基于概率的互联网小组是否比其他模式/抽样策略引入了更多的偏差？(2)在控制样本选择后，邮件和互联网管理模式是否会影响感兴趣的变量和问题回答模式？(3)通过标准有效性测试衡量，调查对每种模式/抽样策略的表现是否一样好？为了解决这些问题，将使用四种处理方法来管理一项关于阿巴拉契亚山脉南部生态系统改善费用支付意愿的调查，其中包括两种抽样策略(小组、随机数字拨号)和两种调查模式(纸张、计算机)和第五种处理方法，即通过随机数字拨号招募的受访者可以在计算机上(通过互联网)或通过邮件进行调查。如果自行选择参加邮件(或互联网)调查的人回答问题的方式与被分配给邮件(或互联网)方式的人不同，这意味着人们的某些特征可能会导致人们对支付方式的偏向意愿。最先进的技术将用于评估不可忽略的无响应(或对不可观测对象的选择)以及包括人口普查区域特征在内的各种框架变量。该项目将更好地了解影响回应倾向的因素，以及回应倾向是否与调查结果相关，重点放在互联网/小组的组合上。美国管理和预算办公室最近发布了指导意见，要求对回复率低于80%的调查进行非回应偏差分析。然而，指导意见没有明确说明如何进行这种分析，而且关于如何进行这种分析以及影响调查答复的因素的调查方法研究相对较少。本项目提出并评估了测试无反应对样本代表性的影响的方法。这些问题也引起了美国环境保护局的关注，美国环境保护局通过国家环境经济中心的竞争性拨款共同资助了这个项目。

Probability-based internet survey panels potentially provide survey responses from a sample representative of the general population at low cost and with the ability to provide complex information treatments. Despite using random digit dialing to recruit the panelists, this technique raises concerns about the representativeness of survey responses given the multiple stages of panelist recruitment and retention. Relatively little also is known about how mode of administration, such as taking a survey on a computer, affects survey response. This research project will pursue answers to the following questions: (1) What nonresponse biases are introduced by mode-of-administration and sampling strategies? Specifically, do probability-based internet panels introduce more bias than other modes/sampling strategies? (2) After controlling for sample selection, do the mail and internet modes of administration affect the variable of interest and question response patterns? (3) Does the survey perform equally well for each mode/sampling strategy as gauged by standard tests of validity? To address these questions, four treatment approaches will be used to administer a survey on the willingness to pay for ecosystem improvements in the Southern Appalachian Mountains, crossing two sampling strategies (panel, random digit dialing) with two survey modes (paper, computer) and a fifth treatment in which respondents recruited through random digit dialing may take the survey on a computer (by internet) or by mail. If people who self-select into a mail (or internet) survey answer differently than people who are assigned to a mail (or internet) mode, this would imply that there are characteristics of people that can lead to biased willingness to pay depending on mode. State-of-the-art techniques will be used for evaluating non-ignorable nonresponse (or selection on unobservables) along with a rich variety of frame variables including Census tract characteristics. The project will provide greater understanding of the factors affecting response propensity and whether response propensity is correlated with the survey outcomes, focusing on the internet/panel combination.The U.S. Office of Management and Budget has recently issued guidance that requires a non-response bias analysis for surveys expecting a response rate less than 80 percent. The guidance does not clarify how such analyses are to be conducted, however, and there is relatively little survey methodology research regarding how such analyses should be conducted and the factors that affect survey response. This project proposes and evaluates methods for testing the effects of nonresponse on sample representativeness. These issues also are of concern to the U.S. Environmental Protection Agency, which co-funded this project through a competitive grant from the National Center for Environmental Economics.