Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA.
Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA.
复制标题
DOI：
10.1371/journal.pone.0279720
复制
发表时间：
2023
期刊：
影响因子：
3.7
通讯作者：
Brauer, Markus
中科院分区：
文献类型：
--
作者：
Douglas, Benjamin D.;Ewell, Patrick J.;Brauer, Markus
随着人类受试者研究中在线数据收集的激增，人们对不专心的调查参与者和非人类受访者（机器人）的存在表示担忧。我们比较了通过五个常用平台收集的数据的质量。数据质量由有意义地回答研究人员问题的参与者（高质量）与仅贡献噪音的参与者（低质量）的百分比表示。我们发现，与MTurk、Qualtrics或本科生样本（即，索纳），多产和CloudResearch的参与者更有可能通过各种注意力检查，提供有意义的答案，遵循指示，记住以前提供的信息，拥有唯一的IP地址和地理位置，并且工作速度足够慢，以便能够阅读所有项目。我们将样本分为高质量和低质量的受访者，并计算了我们为每个高质量受访者支付的成本。Prolific（1.90美元）和CloudResearch（2.00美元）比MTurk（4.36美元）和Qualtrics（8.17美元）便宜。索纳成本为0美元，但收集数据的时间最长。
With the proliferation of online data collection in human-subjects research, concerns have been raised over the presence of inattentive survey participants and non-human respondents (bots). We compared the quality of the data collected through five commonly used platforms. Data quality was indicated by the percentage of participants who meaningfully respond to the researcher’s question (high quality) versus those who only contribute noise (low quality). We found that compared to MTurk, Qualtrics, or an undergraduate student sample (i.e., SONA), participants on Prolific and CloudResearch were more likely to pass various attention checks, provide meaningful answers, follow instructions, remember previously presented information, have a unique IP address and geolocation, and work slowly enough to be able to read all the items. We divided the samples into high- and low-quality respondents and computed the cost we paid per high-quality respondent. Prolific ($1.90) and CloudResearch ($2.00) were cheaper than MTurk ($4.36) and Qualtrics ($8.17). SONA cost $0.00, yet took the longest to collect the data.
登录
查看更多内容
影响因子：
5.7
作者：
Chmielewski, Michael;Kucker, Sarah C.
通讯作者：
Kucker, Sarah C.
影响因子：
5.4
作者：
Peer E;Rothschild D;Gordon A;Evernden Z;Damer E
通讯作者：
Damer E
影响因子：
4.6
作者：
Chandler, Jesse;Sisso, Itay;Shapiro, Danielle
通讯作者：
Shapiro, Danielle
影响因子：
12.6
作者：
Buhrmester, Michael;Kwang, Tracy;Gosling, Samuel D.
通讯作者：
Gosling, Samuel D.
影响因子：
5.7
作者：
Kees, Jeremy;Berry, Christopher;Sheehan, Kim
通讯作者：
Sheehan, Kim