Identifying Careless Responses in Survey Data
Identifying Careless Responses in Survey Data
复制标题
DOI：
10.1037/a0028085
复制
发表时间：
2012-09-01
影响因子：
7
通讯作者：
Craig, S. Bartholomew
中科院分区：
文献类型：
--
作者：
Meade, Adam W.;Craig, S. Bartholomew
当通过匿名互联网调查收集数据时，特别是在强制性参与的条件下（如学生样本），数据质量可能是一个问题。然而，已发表的文献中几乎没有关于检测粗心反应的技术的指导。以前几个潜在的方法已经提出了确定粗心的受访者通过从数据计算的指数，但几乎没有以前的工作已经检查了这些指标之间的关系或数据模式的类型确定的每一个。在2项研究中，我们研究了几种识别粗心反应的方法，包括（a）设计用于检测粗心反应的特殊项目，（B）由典型调查项目的反应形成的反应一致性指数，（c）多变量离群值分析，（d）反应时间，和（e）自我报告的勤奋。结果表明，有两种不同的模式，粗心的反应（随机和非随机），并需要不同的指标来识别这些不同的反应模式。我们还发现，大约10%-12%的本科生完成了一个漫长的调查课程学分被确定为粗心的反应。在研究2中，我们模拟了具有已知随机反应模式的数据，以确定粗心反应的几个指标的功效。我们发现，数据的性质强烈影响的指标，以确定粗心的反应的效力。建议包括使用确定的而不是匿名的响应，在数据收集之前纳入指示的响应项目，以及计算一致性指数和多变量离群值分析，以确保高质量的数据。
When data are collected via anonymous Internet surveys, particularly under conditions of obligatory participation (such as with student samples), data quality can be a concern. However, little guidance exists in the published literature regarding techniques for detecting careless responses. Previously several potential approaches have been suggested for identifying careless respondents via indices computed from the data, yet almost no prior work has examined the relationships among these indicators or the types of data patterns identified by each. In 2 studies, we examined several methods for identifying careless responses, including (a) special items designed to detect careless response, (b) response consistency indices formed from responses to typical survey items, (c) multivariate outlier analysis, (d) response time, and (e) self-reported diligence. Results indicated that there are two distinct patterns of careless response (random and nonrandom) and that different indices are needed to identify these different response patterns. We also found that approximately 10%-12% of undergraduates completing a lengthy survey for course credit were identified as careless responders. In Study 2, we simulated data with known random response patterns to determine the efficacy of several indicators of careless response. We found that the nature of the data strongly influenced the efficacy of the indices to identify careless responses. Recommendations include using identified rather than anonymous responses, incorporating instructed response items before data collection, as well as computing consistency indices and multivariate outlier analysis to ensure high-quality data.