NSF Convergence Accelerator Track F: Misinformation Judgments with Public Legitimacy

要求传播平台采取行动减少错误信息的影响的呼声越来越高，要么将文章和帖子标记为误导性的，要么减少其传播。但对于应该使用什么程序来确定哪些文章和帖子应该接受这种执法行动，几乎没有达成一致意见。平台不愿对其政策和程序完全透明，部分原因是这些程序依赖于人的判断，而人的判断的公共合法性尚未确立。我们需要的是一个清晰的概念基础，让平台能够用“主观基本事实”来描述政策，并对他们的程序执行政策的情况进行公开问责。该项目将开发两项服务。第一种是“黄金套装”服务，每月对一批物品进行抽样，召集陪审团做出判决。陪审团将考虑围绕某一项目收集的证据，并相互商议。然后，每名陪审员将报告一份关于该物品是否以值得采取执法行动的方式误报信息的个人判断。这些个别判决提供了对假设的主观基本事实的部分看法，其定义是，如果所有潜在陪审员都参与类似的陪审团程序，他们将作出的判决的分布。然后，平台可以根据主观基本事实透明地表达它们打算采取的行动。黄金套装将提供一种方法，在事后评估平台的程序执行其政策的情况，建立对程序结果的问责，而无需要求平台公开这些程序的细节。第二个服务是预测API，可用作平台决策过程的一部分。它将为任何内容项目提供对主观基础真相的预测。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

There have been increasingly frequent calls for communication platforms to take action to reduce the impact of misinformation, either by marking articles and posts as misleading or by reducing their distribution. But there is little agreement on what process should be used to determine which articles and posts should receive such enforcement actions. Platforms are reluctant to be fully transparent about their policies and procedures, in part because those procedures depend on human judgments whose public legitimacy is not established. What is needed is a clear conceptual foundation that allows platforms to describe policies in terms of a "subjective ground truth" and be publicly accountable for how well their procedures enact their policies.The project will develop two services. The first is a "golden set" service that, for a sample of items each month, convenes a jury to render judgments. A jury will consider evidence that has been assembled about an item and deliberate with each other. Each juror will then report out an individual judgment about whether the item is misinforming in a way that deserves enforcement action. Those individual judgments provide a partial view of the hypothetical subjective ground truth, which is defined as the distribution of judgments that would be made by all potential jurors if they all participated in a similar jury process. Platforms can then express policies transparently in terms of the actions they intend to take as a function of the subjective ground truth. The golden sets will provide a way to evaluate, after the fact, how well a platform's procedures implement its policies, creating accountability for the outcome of the procedures without requiring the platforms to make details of those procedures public. The second service is a forecasting API that can be used as part of a platform's decision procedures. It will provide, for any content item, a forecast of the subjective ground truth.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.