SaTC: CORE: Medium: Learning Code(s): Community-Centered Design of Automated Content Moderation

在线平台带来了言论自由的好处和坏处：虽然它们帮助我们建立联系和分享想法，但它们也可能为仇恨言论和极端主义提供便利。内容版主致力于执行旨在减轻这些负面行为的社区规则，但由于反复暴露于有毒内容，他们面临着沉重的负担。原则上，使用自然语言处理(NLP)和机器学习(ML)技术的自动化工具可以减轻这一负担。然而，当前的NLP和ML技术可以通过使用微妙和编码的语言来绕过坚定的海报，而使用它们的审核工具通常很难为他们社区的规范、政策和审核实践进行配置。该项目利用社区已经在网上制定和执行不同的言论政策的事实，1)教软件学习版主在现有社区做出的决定的细微差别；2)支持版主，不仅标记内容，而且建议决定并为这些决定提供解释；以及3)提供审计工具，帮助社区成员知道版主是按照规范和政策行事的。在进行这项研究时，该小组将开发工具来支持更健康的在线社区，特别是志愿者领导的社区，方法是加强政策执行，为在线版主(他们通常来自边缘化社区)创造更好的工作条件，为社区政策创造更灵活的软件回应，并支持适应未来对内容审查的监管。为了实现这些目标，跨学科项目团队正在进行周期研究，包括与主持人一起进行经验需求发现研究、开发基于NLP和ML的工具、评估以及对这些工具进行迭代改进。该项目的经验性研究将促进人们对“机器在环”调节(自动化工具作出或支持调节决策)如何影响主持人工作条件和在线参与者体验的了解，并为评估机制提供信息，以衡量ML工具在尊重在线社区政策和确定不成文的社区规范方面的成功。该项目的设计过程将在ML算法方面取得根本性进展，这些算法使用主持人提供的理由从很少的标签中学习，以及改进基于人类理性的机器决策解释。总而言之，这些进步为ML工具产生了新的设计方法，这些方法适应复杂的书面政策并识别不成文的社会规范，以负责任和透明的方式为多个利益相关者服务。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Online platforms bring out the best and worst of free speech: while they help us make connections and share ideas, they can also facilitate hate speech and extremism. Content moderators work to enforce community rules designed to mitigate these negative behaviors, but face a high burden from repeated exposure to toxic content. In principle, automated tools that use natural language processing (NLP) and machine learning (ML) techniques could ease this burden. However, current NLP and ML techniques can be circumvented by determined posters through the use of subtle and coded language, and the moderation tools that use them are often hard for moderators to configure for their community's norms, policies, and moderation practices. This project leverages the fact that communities already make and enforce diverse speech policies online to 1) teach software to learn nuance from the decisions moderators make in existing communities; 2) support moderators by not only flagging content, but also suggesting decisions and providing explanations for those decisions; and 3) provide auditing tools that help community members know that moderators are acting in accordance with norms and policies. In doing this research, the team will develop tools to support healthier online communities, particularly volunteer-led communities, by strengthening policy enforcement, enabling better working conditions for online moderators (who are often from marginalized communities), creating more flexible software responses to community policies, and supporting adaptability to future regulation of content moderation. To achieve these goals, the cross-disciplinary project team is conducting cycles the involve empirical needs-finding studies with moderators, development of NLP and ML-based tools, evaluation, and iterative improvement of those tools. The project's empirical studies will advance knowledge of how ``machine-in-the-loop'' moderation (where automated tools make or support moderation decisions) impacts moderator working conditions and online participant experiences, as well as informing evaluation mechanisms for measuring the success of the ML tools at respecting online community policies and identifying unwritten community norms. The project's design process will make fundamental progress in ML algorithms that learn from few labels using justifications provided by moderators, as well as improving explanations for machine decisions based on human rationales. Together, these advances produce new design methods for ML tools that adapt to complex written policies and identify unwritten social norms, serving multiple stakeholders accountably and transparently.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.