Are poor quality data just random responses?: A crowdsourced study of delay discounting in alcohol use disorder.
Are poor quality data just random responses?: A crowdsourced study of delay discounting in alcohol use disorder.
复制标题
DOI：
10.1037/pha0000549
复制
发表时间：
2022-08
影响因子：
2.3
通讯作者：
Bickel, Warren K.
中科院分区：
文献类型：
--
作者：
Craft, William H.;Tegge, Allison N.;Freitas-Lemos, Roberta;Tomlinson, Devin C.;Bickel, Warren K.
Amazon Mechanical Turk (MTurk) 等众包数据收集方法已在成瘾科学中广泛采用。最近的报告表明 MTurk 的低质量数据有所增加，对研究结果的有效性提出了挑战。然而，缺乏对成瘾相关样本数据质量的实证研究。在这项针对酒精使用障碍 (AUD) 个体的研究中，我们将质量较差的延迟贴现数据与随机生成的数据进行了比较。对先前发布的延迟贴现数据进行了重新分析，比较了包含的、排除的和随机生成的数据样本。采用非系统标准作为数据质量的衡量标准。排除的数据与纳入的样本在统计上存在差异，但与多个指标的随机生成的数据没有差异。此外，在排除的数据中发现了反应偏差。这项研究提供的经验证据表明，AUD 样本中质量较差的延迟贴现数据与随机生成的数据在统计上没有差异，这表明对 MTurk 的数据质量问题在成瘾样本中仍然存在。这些发现支持使用严格的先验定义标准方法来事后删除质量较差的数据。此外，它还强调，使用非系统延迟贴现标准来删除质量较差的数据是严格的，而不仅仅是删除不符合预期理论模型的数据的一种方法。
Crowdsourced methods of data collection such as Amazon Mechanical Turk (MTurk) have been widely adopted in addiction science. Recent reports suggest an increase in poor quality data on MTurk, posing a challenge to the validity of findings. However, empirical investigations of data quality in addiction-related samples are lacking. In this study of individuals with alcohol use disorder (AUD) we compared poor quality delay discounting data to randomly generated data. A reanalysis of prior published delay discounting data was conducted comparing included, excluded, and randomly generated data samples. Non-systematic criteria were implemented as a measure of data quality. The excluded data was statistically different from the included sample but did not differ from randomly generated data on multiple metrics. Moreover, a response bias was identified in the excluded data. This study provides empirical evidence that poor quality delay discounting data in an AUD sample is not statistically different from randomly generated data, suggesting data quality concerns on MTurk persist in addiction samples. These findings support the use of rigorous methods of a priori defined criteria to remove poor quality data post hoc. Additionally, it highlights that the use of non-systematic delay discounting criteria to remove poor quality data is rigorous and not simply a way of removing data that does not conform to an expected theoretical model.
登录
查看更多内容
影响因子：
5.7
作者：
Chmielewski, Michael;Kucker, Sarah C.
通讯作者：
Kucker, Sarah C.
影响因子：
3.9
作者：
Kennedy, Ryan;Clifford, Scott;Winter, Nicholas J. G.
通讯作者：
Winter, Nicholas J. G.
影响因子：
2.3
作者：
Strickland, Justin C.;Stoops, William W.
通讯作者：
Stoops, William W.
影响因子：
1.3
作者：
Du, WJ;Green, L;Myerson, J
通讯作者：
Myerson, J
影响因子：
5.4
作者：
Hydock, Chris
通讯作者：
Hydock, Chris