Wisdom-of-Crowds Approaches for Improving Predictions from Surveys

这项研究项目将调查如何利用人群的智慧来提高调查的准确性。调查是推断当前态度、观点和行为以及预测未来趋势的重要工具。然而，他们的有效性面临威胁，这可能会降低他们推断的准确性。该项目将探索基于人群智慧的两种方法对提高调查准确性的价值。一种方法是利用受访者对自己社交圈的了解，即让受访者预测朋友和家人的行为。另一种方法是评分程序，奖励诚实和仔细的答案。通过对这些方法背后的机制进行细粒度测试，该项目将提供关于如何减少对调查有效性的常见威胁的见解。该项目还将介绍如何捕捉与选举和其他社会趋势有关的意见变化的早期迹象。这些结果可能会改变选举民调的进行方式，并改善选举预测。这个项目的数据将公之于众。调查结果将广泛传播给科学家、调查研究从业者、民调公司、政客、政策制定者、公共教育工作者、记者和普通公众。这项研究项目将研究如何通过纳入受访者对其社交圈的了解并使用贝叶斯真理血清算法对受访者的答案进行评分来提高政治民调的准确性。社交圈的期望值问题询问受访者他们的家人和朋友会如何投票。贝叶斯真相血清是一种评分方法，可用于激励提供诚实和仔细答案的参与者，并有选择地对他们进行加权。调查人员将评估这两种方法的结合如何克服民调有效性面临的四个公认威胁：样本偏差、回答粗心、社会合意回答和社会动态。将进行纵向调查，并以全国范围内的概率抽样。这项调查将包括2020年美国总统大选前的三波和大选后的一波。调查人员将把传统的调查问题与群体智慧方法进行比较。基准将包括其他同时进行的民意调查和预测模型，最终包括实际选举结果和选举后调查。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will investigate how survey accuracy can be improved using the wisdom of crowds. Surveys are important tools for making inferences about current attitudes, opinions, and behaviors as well as making forecasts about future trends. However, they are facing threats to their validity that can decrease the accuracy of their inferences. The project will explore the value of two methods based on the wisdom of crowds for improving survey accuracy. One method entails using respondents' knowledge of their social circles, that is, asking respondents to predict the behavior of friends and family. The other method is a scoring procedure that rewards honest and careful answers. By conducting fine-grained tests of the mechanisms underlying these methods, the project will provide insights regarding how to reduce common threats to the validity of surveys. The project also will inform ways to capture the early indications of opinion change relevant for elections and other societal trends. The results could change the way election polls are conducted and improve election predictions. Data from this project will be made publicly available. The results will be disseminated broadly to scientists, practitioners in survey research, polling companies, politicians, policy makers, public educators, journalists, and the general public.This research project will examine how political polling accuracy can be improved by incorporating respondents' knowledge of their social-circles and by scoring respondents' answers with the Bayesian Truth Serum algorithm. Social-circle expectation questions ask respondents how their family and friends will vote. Bayesian Truth Serum is a scoring method that can be used to incentivize and selectively weight participants who provide honest and careful answers. The investigators will assess how the combination of both methods can overcome four recognized threats to polling validity: biased samples, careless responding, socially desirable responding, and social dynamics. A longitudinal survey with a probabilistic national sample will be conducted. The survey will consist of three waves before the 2020 U.S. presidential election and one wave after the election. The investigators will compare traditional survey questions with wisdom-of-crowd methods. Benchmarks will include other concurrent polls and forecasting models and eventually the actual election results and a post-election survey.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.