Collaborative Research: Decomposing Interviewer Variance in Standardized and Conversational Interviewing

合作研究：分解标准化和对话式访谈中访谈者的差异

• 批准号: 1323636
• 负责人: Frauke Kreuter
• 金额: $ 3.58万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1323636&HistoricalAwards=false
• 关键词: Collaborative; Research; Decomposing; Interviewer; Variance

Standardized interviewing procedures require survey interviewers to read questions as worded and provide only neutral or non-directive probes in response to questions from survey respondents. Even though many major surveys in the government, non-profit, and private sectors use standardized interviewing in an effort to minimize the effects of interviewers on data quality, a substantial body of research has indicated that interviewers using standardized interviewing still influence the responses provided by individuals. This variability among interviewers (or interviewer variance) in the types of survey responses collected reduces the precision of survey estimates and therefore has direct cost implications for survey data collection. Conversational interviewing is known to handle respondent clarification requests in a more effective manner. Interviewers are trained to read questions as worded initially and then say whatever is required to help respondents understand the questions. Despite existing research showing that conversational interviewing produces noticeable decreases in the measurement error bias of survey estimates, survey researchers (and government agencies in particular) have been hesitant to employ it in practice, in part because of increased questionnaire administration time but also due to the fear of increased interviewer effects on the survey data due to the conversational style. This research project will compare the interviewer variance, bias, and mean squared error arising in a variety of survey estimates from these two face-to-face interviewing techniques. It will decompose the total interviewer variance introduced by each technique into measurement error variance (i.e., variance among interviewers in systematic measurement errors) and nonresponse error variance (i.e., variance among interviewers in the types of individuals recruited for the survey). To meet these research objectives, this study will select a large random sample of persons from a unique economic database that contains known values for selected economic characteristics of interest. The researchers will assign random subsets of this sample to professional interviewers who have been randomly assigned to receive training in one of the two interviewing techniques. The two groups of interviewers will then administer a face-to-face survey to their assigned subsamples, collecting information on their economic characteristics. Analyses of the collected data will employ innovative statistical modeling techniques that enable comparisons of the interviewer variance, bias, and mean squared error values for a variety of economic survey estimates across the two groups of interviewers as well as decompositions of the total interviewer variance in each group into the two aforementioned sources of variance among interviewers. These analyses will provide survey researchers with the first empirical evidence of differences in these two techniques in terms of the overall quality of survey estimates and will uncover sources of the interviewer effects that can arise when using the two interviewing styles.As a scientific field, survey methodology, which studies the science of survey data collection, is still relatively nascent, but billions of dollars in public resources are dedicated to survey data collection every year. This study will provide survey researchers with a more complete set of empirical evidence that will enable informed decisions about the face-to-face interviewing style that will yield survey estimates with the highest overall quality. The resulting knowledge regarding which interviewing technique produces higher-quality estimates therefore will lead to higher-quality information being collected in surveys that employ face-to-face interviewing and higher-quality decisions being made by policy makers who use survey data.

标准化的访谈程序要求调查采访者按照文字来阅读问题，并且在回答调查对象的问题时只提供中立或非指导性的询问。尽管政府、非营利组织和私营部门的许多主要调查都使用标准化访谈，以尽量减少访谈者对数据质量的影响，但大量研究表明，使用标准化访谈的访谈者仍然会影响个人提供的回答。在收集的调查回答类型中，采访者之间的这种可变性（或采访者差异）降低了调查估计的精度，因此对调查数据收集有直接的成本影响。会话式访谈以一种更有效的方式处理被调查者澄清要求。采访者经过培训，一开始会按字面意思阅读问题，然后说一些帮助被访者理解问题所需的话。尽管现有的研究表明，会话式访谈可以显著降低调查估计的测量误差偏差，但调查研究人员（尤其是政府机构）一直在犹豫是否将其应用于实践，部分原因是增加了问卷管理时间，但也因为担心由于会话风格会增加访谈者对调查数据的影响。本研究项目将比较这两种面对面访谈技术在各种调查估计中产生的访谈者方差、偏差和均方误差。它将每种技术引入的访谈者总方差分解为测量误差方差（即访谈者在系统测量误差中的方差）和非响应误差方差（即访谈者在被调查的个体类型中的方差）。为了实现这些研究目标，本研究将从一个独特的经济数据库中选择一个随机的大样本，该数据库包含了所选经济特征的已知值。研究人员将随机分配这些样本的子集给专业面试官，这些面试官被随机分配接受两种面试技巧之一的培训。然后，两组采访者将对其指定的子样本进行面对面的调查，收集有关其经济特征的信息。对收集到的数据的分析将采用创新的统计建模技术，可以对两组采访者的各种经济调查估计的采访者方差、偏差和均方误差值进行比较，并将每组采访者的总方差分解为上述两种采访者的方差来源。这些分析将为调查研究人员提供第一个经验证据，证明这两种技术在调查估计的总体质量方面存在差异，并将揭示使用这两种访谈风格时可能产生的访谈者效应的来源。作为一个科学领域，调查方法论研究的是调查数据收集的科学，它还处于相对新兴的阶段，但每年都有数十亿美元的公共资源用于调查数据收集。这项研究将为调查研究人员提供一套更完整的经验证据，这将使面对面访谈风格的明智决策成为可能，从而产生具有最高整体质量的调查估计。由此产生的关于哪种访谈技术产生更高质量估计的知识将导致在采用面对面访谈的调查中收集到更高质量的信息，并由使用调查数据的政策制定者做出更高质量的决策。