SaTC: CORE: Medium: Learning Code(s): Community-Centered Design of Automated Content Moderation

SaTC：核心：媒介：学习代码：以社区为中心的自动内容审核设计

• 批准号: 2131508
• 负责人: Katherine Shilton
• 金额: $ 78万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2131508&HistoricalAwards=false
• 关键词: SaTC; CORE; Medium; Learning; Code

Online platforms bring out the best and worst of free speech: while they help us make connections and share ideas, they can also facilitate hate speech and extremism. Content moderators work to enforce community rules designed to mitigate these negative behaviors, but face a high burden from repeated exposure to toxic content. In principle, automated tools that use natural language processing (NLP) and machine learning (ML) techniques could ease this burden. However, current NLP and ML techniques can be circumvented by determined posters through the use of subtle and coded language, and the moderation tools that use them are often hard for moderators to configure for their community's norms, policies, and moderation practices. This project leverages the fact that communities already make and enforce diverse speech policies online to 1) teach software to learn nuance from the decisions moderators make in existing communities; 2) support moderators by not only flagging content, but also suggesting decisions and providing explanations for those decisions; and 3) provide auditing tools that help community members know that moderators are acting in accordance with norms and policies. In doing this research, the team will develop tools to support healthier online communities, particularly volunteer-led communities, by strengthening policy enforcement, enabling better working conditions for online moderators (who are often from marginalized communities), creating more flexible software responses to community policies, and supporting adaptability to future regulation of content moderation. To achieve these goals, the cross-disciplinary project team is conducting cycles the involve empirical needs-finding studies with moderators, development of NLP and ML-based tools, evaluation, and iterative improvement of those tools. The project's empirical studies will advance knowledge of how ``machine-in-the-loop'' moderation (where automated tools make or support moderation decisions) impacts moderator working conditions and online participant experiences, as well as informing evaluation mechanisms for measuring the success of the ML tools at respecting online community policies and identifying unwritten community norms. The project's design process will make fundamental progress in ML algorithms that learn from few labels using justifications provided by moderators, as well as improving explanations for machine decisions based on human rationales. Together, these advances produce new design methods for ML tools that adapt to complex written policies and identify unwritten social norms, serving multiple stakeholders accountably and transparently.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在线平台带来了最好和最坏的言论自由：虽然它们帮助我们建立联系和分享想法，但它们也可以促进仇恨言论和极端主义。内容版主致力于执行旨在减轻这些负面行为的社区规则，但由于反复接触有毒内容而面临沉重的负担。 原则上，使用自然语言处理（NLP）和机器学习（ML）技术的自动化工具可以减轻这一负担。 然而，当前的NLP和ML技术可以通过使用微妙和编码的语言来规避，并且使用它们的审核工具通常很难让版主根据其社区的规范，政策和审核实践进行配置。 该项目利用社区已经在线制定和执行各种言论政策的事实，1）教软件从现有社区中的主持人所做的决定中学习细微差别; 2）不仅通过标记内容来支持主持人，而且还建议决定并为这些决定提供解释;以及3）提供审计工具，帮助社区成员了解版主的行为符合规范和政策。 在进行这项研究时，该团队将开发工具来支持更健康的在线社区，特别是志愿者领导的社区，通过加强政策执行，为在线版主（通常来自边缘化社区）创造更好的工作条件，创建更灵活的软件响应社区政策，并支持适应未来的内容审核监管。 为了实现这些目标，跨学科项目团队正在进行周期性的研究，包括与主持人进行实证需求发现研究，开发基于NLP和ML的工具，评估以及这些工具的迭代改进。该项目的实证研究将进一步了解“机器在环”审核（自动化工具做出或支持审核决策）如何影响审核员的工作条件和在线参与者体验，并为评估机制提供信息，以衡量ML工具在尊重在线社区政策和识别不成文的社区规范方面的成功。该项目的设计过程将在机器学习算法方面取得根本性进展，这些算法使用主持人提供的理由从少数标签中学习，并改进基于人类理性的机器决策解释。这些进步共同为机器学习工具创造了新的设计方法，这些工具可以适应复杂的书面政策，识别不成文的社会规范，以负责任和透明的方式为多个利益相关者服务。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。