Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments
Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments
复制标题
DOI：
10.3758/s13423-012-0296-9
复制
发表时间：
2012-10-01
影响因子：
3.5
通讯作者：
Wilmer, Jeremy B.
中科院分区：
文献类型：
--
作者：
Germine, Laura;Nakayama, Ken;Wilmer, Jeremy B.
随着互联网的日益成熟和普及，行为研究正处于一场革命的风口浪尖，这场革命将为人口抽样带来计算机对刺激控制和测量的影响。然而，这仍然是一个常见的假设，即来自自我选择的Web样本的数据必须涉及参与者数量和数据质量之间的权衡。对于基于性能的认知和感知测量，特别是那些定时或涉及复杂刺激的测量，对数据质量的担忧加剧。在无报酬的匿名参与者的参与动机未知的实验中，由于整体表现下降，表现的可变性增加或测量噪音增加，减少的好奇心或缺乏焦点可能会产生难以解释的结果。在这里，我们解决了一系列认知和感知测试中的数据质量问题。对于三个关键的性能指标，平均性能，性能方差和内部可靠性，从自我选择的Web样本的结果并没有系统地从传统的招聘和/或实验室测试的样本。这些发现表明，从无报酬、匿名、无监督、自我选择的参与者那里收集数据并不一定会降低数据质量，即使对于要求苛刻的认知和感知实验也是如此。
With the increasing sophistication and ubiquity of the Internet, behavioral research is on the cusp of a revolution that will do for population sampling what the computer did for stimulus control and measurement. It remains a common assumption, however, that data from self-selected Web samples must involve a trade-off between participant numbers and data quality. Concerns about data quality are heightened for performance-based cognitive and perceptual measures, particularly those that are timed or that involve complex stimuli. In experiments run with uncompensated, anonymous participants whose motivation for participation is unknown, reduced conscientiousness or lack of focus could produce results that would be difficult to interpret due to decreased overall performance, increased variability of performance, or increased measurement noise. Here, we addressed the question of data quality across a range of cognitive and perceptual tests. For three key performance metrics-mean performance, performance variance, and internal reliability-the results from self-selected Web samples did not differ systematically from those obtained from traditionally recruited and/or lab-tested samples. These findings demonstrate that collecting data from uncompensated, anonymous, unsupervised, self-selected participants need not reduce data quality, even for demanding cognitive and perceptual experiments.