Collaborative Research: Decomposing Interviewer Variance in Standardized and Conversational Interviewing

合作研究：分解标准化和对话式访谈中访谈者的差异

• 批准号: 1324689
• 负责人: Brady West
• 金额: $ 37.42万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1324689&HistoricalAwards=false
• 关键词: Collaborative; Research; Decomposing; Interviewer; Variance

Standardized interviewing procedures require survey interviewers to read questions as worded and provide only neutral or non-directive probes in response to questions from survey respondents. Even though many major surveys in the government, non-profit, and private sectors use standardized interviewing in an effort to minimize the effects of interviewers on data quality, a substantial body of research has indicated that interviewers using standardized interviewing still influence the responses provided by individuals. This variability among interviewers (or interviewer variance) in the types of survey responses collected reduces the precision of survey estimates and therefore has direct cost implications for survey data collection. Conversational interviewing is known to handle respondent clarification requests in a more effective manner. Interviewers are trained to read questions as worded initially and then say whatever is required to help respondents understand the questions. Despite existing research showing that conversational interviewing produces noticeable decreases in the measurement error bias of survey estimates, survey researchers (and government agencies in particular) have been hesitant to employ it in practice, in part because of increased questionnaire administration time but also due to the fear of increased interviewer effects on the survey data due to the conversational style. This research project will compare the interviewer variance, bias, and mean squared error arising in a variety of survey estimates from these two face-to-face interviewing techniques. It will decompose the total interviewer variance introduced by each technique into measurement error variance (i.e., variance among interviewers in systematic measurement errors) and nonresponse error variance (i.e., variance among interviewers in the types of individuals recruited for the survey). To meet these research objectives, this study will select a large random sample of persons from a unique economic database that contains known values for selected economic characteristics of interest. The researchers will assign random subsets of this sample to professional interviewers who have been randomly assigned to receive training in one of the two interviewing techniques. The two groups of interviewers will then administer a face-to-face survey to their assigned subsamples, collecting information on their economic characteristics. Analyses of the collected data will employ innovative statistical modeling techniques that enable comparisons of the interviewer variance, bias, and mean squared error values for a variety of economic survey estimates across the two groups of interviewers as well as decompositions of the total interviewer variance in each group into the two aforementioned sources of variance among interviewers. These analyses will provide survey researchers with the first empirical evidence of differences in these two techniques in terms of the overall quality of survey estimates and will uncover sources of the interviewer effects that can arise when using the two interviewing styles.As a scientific field, survey methodology, which studies the science of survey data collection, is still relatively nascent, but billions of dollars in public resources are dedicated to survey data collection every year. This study will provide survey researchers with a more complete set of empirical evidence that will enable informed decisions about the face-to-face interviewing style that will yield survey estimates with the highest overall quality. The resulting knowledge regarding which interviewing technique produces higher-quality estimates therefore will lead to higher-quality information being collected in surveys that employ face-to-face interviewing and higher-quality decisions being made by policy makers who use survey data.

标准化的访谈程序要求调查访谈员按问题的措辞阅读问题，并在回答调查受访者的问题时只提供中立或非指导性的调查。 尽管政府、非营利组织和私营部门的许多主要调查都使用标准化访谈，以尽量减少访谈者对数据质量的影响，但大量研究表明，使用标准化访谈的访谈者仍然会影响个人提供的回答。 访谈者之间在收集的调查答复类型方面的这种差异（或访谈者差异）降低了调查估计数的准确性，因此对调查数据收集产生直接的费用影响。 众所周知，对话式面试可以更有效地处理回答者的澄清请求。 面试官接受培训，首先阅读问题的措辞，然后说出任何需要帮助受访者理解问题的内容。 尽管现有的研究表明，对话式采访产生显着减少测量误差偏差的调查估计，调查研究人员（特别是政府机构）一直犹豫，在实践中使用它，部分原因是增加问卷管理时间，但也由于担心增加采访者的影响，由于调查数据的对话风格。 这个研究项目将比较这两种面对面访谈技术在各种调查估计中产生的访谈者方差、偏差和均方误差。 它将把每种技术引入的总访谈者方差分解为测量误差方差（即，系统测量误差中的访谈者之间的方差）和无应答误差方差（即，在为调查招募的个人的类型中的采访者之间的差异）。 为了实现这些研究目标，本研究将从一个独特的经济数据库中选择一个大的随机样本，该数据库包含选定的感兴趣的经济特征的已知值。 研究人员将随机分配这些样本的子集给专业面试官，他们被随机分配接受两种面试技巧之一的培训。 然后，两组访问员会向其指定的次样本进行面对面的调查，收集有关其经济特征的资料。 对所收集数据的分析将采用创新的统计建模技术，这些技术可以比较两组访谈者的各种经济调查估计的访谈者方差、偏差和均方误差值，以及将每组中的总访谈者方差分解为访谈者之间的上述两个方差来源。 这些分析将为调查研究人员提供第一个实证证据，证明这两种技术在调查估计的整体质量方面的差异，并将揭示使用两种采访风格时可能出现的采访者效应的来源。作为一个科学领域，调查方法学研究调查数据收集的科学，仍然相对较新，但每年用于调查数据收集的公共资源达数十亿美元。 这项研究将为调查研究人员提供一套更完整的经验证据，使他们能够对面对面的访谈风格做出明智的决定，从而获得具有最高整体质量的调查估计。 因此，了解哪种访谈技术可产生更高质量的估计数，将有助于在采用面对面访谈的调查中收集更高质量的信息，并有助于决策者利用调查数据作出更高质量的决定。