The collective intelligence of random small crowds: A partial replication of Kosinski et al. (2012)
The collective intelligence of random small crowds: A partial replication of Kosinski et al. (2012)
复制标题
随机小群体的集体智慧：科辛斯基等人的部分复制。
DOI：
10.1017/s1930297500002941
复制
发表时间：
2019
影响因子：
2.5
通讯作者：
M. Burgman
中科院分区：
文献类型：
--
作者：
A. Vercammen;Yan Ji;M. Burgman
我们在流行的众包平台Mechanical Turk上研究了响应冗余成本和输出质量增益之间的权衡，作为Kosinski et al.（2012）的部分复制，Kosinski et al.（2012）通过多数投票聚合多个响应来证明性能的显着改善。我们从一个经过验证的智力测试中提交了单个项目作为人类智力任务（HIT），并汇总了由1到24名工人组成的“虚拟组”的响应。虽然最初的研究依赖于在一系列实验条件下相对少量的响应，但我们从大量的HIT中随机和独立地取样，只关注组大小的主要影响。我们发现，平均而言，一组六个MTurker的集体智商比一般人群的平均值高出一个标准差，从而证明了“群体智慧”效应。群体规模和集体智商之间的关系的特点是收益递减，这表明中等规模的群体提供了最好的投资回报。我们还分析了一小部分完成了所有60个测试项目的工人的表现，从而可以直接比较一个群体的集体智商和其成员的个人智商。这表明，随机选择的组的总体表现等同于组内表现最好的个人的表现。我们的研究结果支持这样的想法，即大量的智力能力可以通过众包获得，取决于任务请求中内置的适度冗余。
We examined the trade-off between the cost of response redundancy and the gain in output quality on the popular crowdsourcing platform Mechanical Turk, as a partial replication of Kosinski et al. (2012) who demonstrated a significant improvement in performance by aggregating multiple responses through majority vote. We submitted single items from a validated intelligence test as Human Intelligence Tasks (HITs) and aggregated the responses from “virtual groups” consisting of 1 to 24 workers. While the original study relied on resampling from a relatively small number of responses across a range of experimental conditions, we randomly and independently sampled from a large number of HITs, focusing only on the main effect of group size. We found that – on average – a group of six MTurkers has a collective IQ one standard deviation above the mean for the general population, thus demonstrating a “wisdom of the crowd” effect. The relationship between group size and collective IQ was characterised by diminishing returns, suggesting moderately sized groups provide the best return on investment. We also analysed performance of a smaller subset of workers who had each completed all 60 test items, allowing for a direct comparison between a group’s collective IQ and the individual IQ of its members. This demonstrated that randomly selected groups collectively equalled the performance of the best-performing individual within the group. Our findings support the idea that substantial intellectual capacity can be gained through crowdsourcing, contingent on moderate redundancy built into the task request.
石井，N.；A.W.M.辛普森；C.C.阿什利：科学。
DOI：
--
发表时间：
--
期刊：
影响因子：
--
作者：
通讯作者：
--