EAGER: Opinion Spam in Digital Rulemaking: Techniques, Effects, and Interventions

EAGER：数字规则制定中的意见垃圾：技术、效果和干预措施

• 批准号: 2232169
• 负责人: Matthew Jensen
• 金额: $ 30万
• 依托单位: University of Oklahoma Norman Campus
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2232169&HistoricalAwards=false
• 关键词: EAGER; Opinion; Spam; Digital; Rulemaking

One of the pillars of representative government in the United States is people's ability to participate in setting rules and regulations. Regulatory agencies are required (with some exceptions) to solicit comments from various publics (e.g., general citizenry, affected organizations, interest groups) to learn about the potential consequences of proposed rule changes. To extend participation and reduce costs, the commenting process has been digitized and often takes place through the internet. However, digitization opened the door to opinion spam (e.g., mass, computer-generated, or fraudulent comments) that may undermine the rulemaking process by deceiving agency evaluators and manipulating what citizens' actual attitudes are regarding proposed regulations. Opinion spam complicates the evaluation that agencies must perform in setting rules and threatens the legitimacy of the rulemaking process in the eyes of stakeholders. This project investigates ways in which opinion spam might be prevented and provides evidence regarding which techniques are most effective, thereby preserving (or potentially restoring) public trust in digital rulemaking. In three phases, this project examines threats to digital rulemaking and tests mitigation approaches to reduce opinion spam. Phase 1 includes a series of interviews with submitters of comments, comment evaluators, and scholarly experts on mis/disinformation to gauge how these groups conceive of opinion spam and its prevalence in commenting discourse and to uncover potential interventions that may limit the submission of opinion spam or help agencies detect its submission. Phase 2 includes generating machine learning datasets (e.g., legitimate, fictitious, and automated text replacement or text recombination comments) and models for distinguishing fictitious/artificial comments from legitimate comments. Phase 3 integrates the findings from the prior phases and includes selecting several viable opinion-spam mitigating strategies and testing their efficacy in randomized, controlled experiments. This multi-method, interdisciplinary investigation contributes to the theory of coordinated influence campaigns. The project develops a syntax-aware deep learning model for detecting fictitious comments and helps determine which mitigation approaches work better for reducing opinion spam.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

美国代议制政府的支柱之一是人民参与制定规章制度的能力。要求监管机构（除某些例外情况外）征求各种公众的意见（例如，一般公民、受影响的组织、利益集团），以了解拟议规则修改的潜在后果。为了扩大参与和降低成本，评论过程已经数字化，通常通过互联网进行。然而，数字化打开了垃圾意见的大门（例如，大量的、计算机生成的或欺骗性的评论），这些评论可能通过欺骗机关评估人员和操纵公民对拟议法规的实际态度而破坏规则制定过程。意见垃圾使行政机关在制定规则时必须进行的评估复杂化，并威胁到利益相关者眼中规则制定过程的合法性。该项目调查了可能防止意见垃圾邮件的方法，并提供了有关哪些技术最有效的证据，从而保持（或可能恢复）公众对数字规则制定的信任。 分三个阶段，该项目研究了数字规则制定的威胁，并测试了减少意见垃圾的缓解方法。第一阶段包括对评论提交者、评论评估者和错误/虚假信息方面的学术专家进行一系列访谈，以评估这些群体如何看待意见垃圾及其在评论话语中的流行程度，并发现可能限制意见垃圾提交或帮助机构检测其提交的潜在干预措施。阶段2包括生成机器学习数据集（例如，合法的、虚构的和自动的文本替换或文本重组注释）以及用于区分虚构/虚构注释与合法注释的模型。第3阶段整合了前几个阶段的研究结果，包括选择几种可行的意见垃圾邮件缓解策略，并在随机对照实验中测试其有效性。这种多方法，跨学科的调查有助于协调影响力运动的理论。该项目开发了一个语法感知的深度学习模型，用于检测虚假评论，并帮助确定哪些缓解方法更好地减少意见垃圾邮件。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。