Assessing and overcoming participant dishonesty in online data collection
Assessing and overcoming participant dishonesty in online data collection
复制标题
DOI：
10.3758/s13428-017-0984-5
复制
发表时间：
2018-08-01
影响因子：
5.4
通讯作者：
Hydock, Chris
中科院分区：
文献类型：
--
作者：
Hydock, Chris
众包服务，如MTurk，为研究人员开放了大量的参与者。不幸的是，很难自信地获得与给定的人口统计学，心理学或行为维度相匹配的样本。这个问题的存在是因为人们对个人参与者的信息知之甚少，而且一些参与者出于经济奖励的目的而歪曲自己的身份。尽管在线工作者通常不会表现出高于平均水平的不诚实，但当研究人员公开要求只有特定人群参与在线研究时，相当一部分人会歪曲他们的身份。在这项研究中，一个拟议的系统进行了测试，研究人员可以使用它来快速，公平，轻松地筛选参与者的任何维度。与公开请求相比，所报告的系统导致参与者虚报的情况显著减少（接近零）。虚假陈述的测试是通过使用过去参与者记录的大型数据库（45，000名独立工作人员）进行的。这项研究提出并测试了在线数据收集日益普遍的做法的一个重要工具。
Crowdsourcing services, such as MTurk, have opened a large pool of participants to researchers. Unfortunately, it can be difficult to confidently acquire a sample that matches a given demographic, psychographic, or behavioral dimension. This problem exists because little information is known about individual participants and because some participants are motivated to misrepresent their identity with the goal of financial reward. Despite the fact that online workers do not typically display a greater than average level of dishonesty, when researchers overtly request that only a certain population take part in an online study, a nontrivial portion misrepresent their identity. In this study, a proposed system is tested that researchers can use to quickly, fairly, and easily screen participants on any dimension. In contrast to an overt request, the reported system results in significantly fewer (near zero) instances of participant misrepresentation. Tests for misrepresentations were conducted by using a large database of past participant records (45,000 unique workers). This research presents and tests an important tool for the increasingly prevalent practice of online data collection.