NSF Convergence Accelerator Track F: Misinformation Judgments with Public Legitimacy

越来越多的人呼吁传播平台采取行动，通过将文章和帖子标记为误导性或减少其传播，减少错误信息的影响。但是，对于应该使用什么程序来确定哪些文章和帖子应该受到这种执法行动，人们几乎没有达成一致意见。平台不愿意完全透明其政策和程序，部分原因是这些程序依赖于人类的判断，而这些判断的公共合法性尚未建立。我们需要的是一个清晰的概念基础，使平台能够以“主观的基本事实”来描述政策，并对它们的程序如何有效地执行政策承担公共责任。第一种是“黄金套装”服务，即每个月对一批物品进行抽样，召集陪审团作出判决。陪审团将考虑关于一个项目的证据，并相互商议。然后，每个陪审员将报告一个关于该项目是否以一种值得采取执法行动的方式误导的个人判断。这些个人判断提供了假设的主观事实的部分观点，这被定义为所有潜在陪审员如果他们都参加了类似的陪审团过程，将做出的判断的分布。然后，平台可以根据主观基础事实的功能，透明地表达他们打算采取的行动。黄金集将提供一种事后评估平台程序执行其政策的方式，为程序的结果建立问责制，而不要求平台公开这些程序的细节。第二个服务是预测API，可以用作平台决策过程的一部分。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

There have been increasingly frequent calls for communication platforms to take action to reduce the impact of misinformation, either by marking articles and posts as misleading or by reducing their distribution. But there is little agreement on what process should be used to determine which articles and posts should receive such enforcement actions. Platforms are reluctant to be fully transparent about their policies and procedures, in part because those procedures depend on human judgments whose public legitimacy is not established. What is needed is a clear conceptual foundation that allows platforms to describe policies in terms of a "subjective ground truth" and be publicly accountable for how well their procedures enact their policies.The project will develop two services. The first is a "golden set" service that, for a sample of items each month, convenes a jury to render judgments. A jury will consider evidence that has been assembled about an item and deliberate with each other. Each juror will then report out an individual judgment about whether the item is misinforming in a way that deserves enforcement action. Those individual judgments provide a partial view of the hypothetical subjective ground truth, which is defined as the distribution of judgments that would be made by all potential jurors if they all participated in a similar jury process. Platforms can then express policies transparently in terms of the actions they intend to take as a function of the subjective ground truth. The golden sets will provide a way to evaluate, after the fact, how well a platform's procedures implement its policies, creating accountability for the outcome of the procedures without requiring the platforms to make details of those procedures public. The second service is a forecasting API that can be used as part of a platform's decision procedures. It will provide, for any content item, a forecast of the subjective ground truth.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.