Collective intelligence improves probabilistic diagnostic assessments
Collective intelligence improves probabilistic diagnostic assessments
复制标题
DOI：
10.1515/dx-2022-0090
复制
发表时间：
2023-02-20
期刊：
影响因子：
3.5
通讯作者：
Dell, Michael S.
中科院分区：
文献类型：
--
作者：
Stehouwer, Nathan R.;Torrey, Keith W.;Dell, Michael S.
目标：集体智慧，“群众的智慧”，旨在通过聚合多个个体输入来提高判断的质量。在这里，我们评估成功的集体智慧的策略，适用于概率诊断judgment.Methods：我们比较了两个系列的临床病例需要概率诊断评估，或“预测”的个人和集体智慧判断的表现。我们使用Brier分数评估预测的质量，该分数将预测与观察到的outcomes.Results进行比较：在这两组案例中，集体智慧的答案几乎优于每一个预测者或团队。集体智慧的改进表现是由改进的分辨率和概率评估的校准所介导的。在一项二次分析中，研究了不同数量的个体输入对来自两个不同数据源的集体智慧答案的影响，在两个数据集中发现了几乎相同的曲线，当平均两个独立输入时，显示出11-12%的改善，4个独立输入平均15%的改善，以及随着个体输入数量的进一步增加而出现的小幅增量改善。结论：我们的研究结果表明，应用集体智慧策略的概率诊断预测是一个很有前途的方法，以提高诊断的准确性和减少诊断错误。
Objectives: Collective intelligence, the "wisdom of the crowd," seeks to improve the quality of judgments by aggregating multiple individual inputs. Here, we evaluate the success of collective intelligence strategies applied to probabilistic diagnostic judgments.Methods: We compared the performance of individual and collective intelligence judgments on two series of clinical cases requiring probabilistic diagnostic assessments, or "forecasts". We assessed the quality of forecasts using Brier scores, which compare forecasts to observed outcomes.Results: On both sets of cases, the collective intelligence answers outperformed nearly every individual forecaster or team. The improved performance by collective intelligence was mediated by both improved resolution and calibration of probabilistic assessments. In a secondary analysis looking at the effect of varying number of individual inputs in collective intelligence answers from two different data sources, nearly identical curves were found in the two data sets showing 11-12% improvement when averaging two independent inputs, 15% improvement averaging four independent inputs, and small incremental improvements with further increases in number of individual inputs.Conclusions: Our results suggest that the application of collective intelligence strategies to probabilistic diagnostic forecasts is a promising approach to improve diagnostic accuracy and reduce diagnostic error.