Natural language processing for detecting toxic, abusive, and hateful language online

从人工智能和虚拟助手到社交媒体和推荐系统，数字技术提供了令人难以置信的力量。以对个人和社会都有利的方式部署这些技术是一项紧迫的挑战。在主流和社交媒体中，内容提供商欢迎反馈；然而，这种反馈可能是“有毒的”：恶意的、辱骂的或冒犯的。网上的有毒评论和帖子是那些打算造成伤害的评论和帖子。它们可能采取人身攻击、辱骂、骚扰、威胁的形式，可能包括亵渎、淫秽或贬损语言，其中仇恨言论是最极端的。在过去的几年里，我仔细研究了在线新闻评论，并开发了自然语言处理(NLP)方法来分析它们。我的长期研究计划开发了强大的文本分类方法，用于情感分析、错误信息检测和内容审核等任务。在接下来的几年里，我的SFU实验室，话语处理实验室，将继续在线研究有毒语言，开发自动检测毒性的方法和算法。我们识别建设性评论的工作，即那些对在线讨论做出积极贡献的评论，为如何自动分类非建设性和有毒评论提供了极好的洞察力。目前检测在线毒性的方法要么基于一般文本特征(字长、文本长度、大写和标点符号)，要么基于可能导致冒犯的单词列表。机器学习方法(有监督的、半监督的或基于神经网络的)依赖于大量的注释数据集，但许多研究表明，这种方法经常失败，因为语言中的消极可能通过隐喻和其他修辞手段被包裹在积极的单词中。包括我们自己在内的研究发现，准确识别和过滤有毒内容需要多学科的视角，利用对语言学的深刻理解，以及自然语言处理和机器学习的当前方法。为了解决自动检测有毒评论方面的现有空白，我计划在未来五年：(目标1)研究隐喻和其他自古以来广为人知的修辞手法(委婉语、长篇大论、夸张、讽刺)如何传达有毒语言。然后我将开发(目标2)一个自动检测修辞格的系统，我将把它集成到(目标3)一个新的内容审核平台中。这项工作的结果将调动对评价性语言和媒体在公共话语中的作用感兴趣的学者的研究，包括语言学家、计算语言学家以及传播和媒体研究人员。在媒体机构、社交媒体平台和公众关注网络滥用、虚假信息以及数字技术在政治和社会中的作用之际，这个项目来得正是时候，将对公共话语做出重要贡献。

Digital technologies offer incredible power, from artificial intelligence and virtual assistants to social media and recommendation systems. Deploying such technologies in a manner beneficial to both individuals and society is a pressing challenge. In mainstream and social media, content providers welcome feedback; such feedback, however, may be `toxic': malicious, abusive, or offensive. Toxic comments and posts online are those that intend to cause harm. They may take the form of personal attacks, abuse, harassment, threats, and may include profane, obscene, or derogatory language, with hate speech being the most extreme. In the last few years, I have closely studied online news comments and developed natural language processing (NLP) methods to analyze them. My long-term program of research develops robust methods for text classification in tasks such as sentiment analysis, misinformation detection, and content moderation. In the next few years, my SFU laboratory, the Discourse Processing Lab, will continue to study toxic language online, to develop methods and algorithms to detect toxicity automatically. Our work identifying constructive comments, those that contribute positively to an online discussion, has provided excellent insight for how to automatically classify non-constructive and toxic comments. Current approaches to detecting online toxicity are based either on general text characteristics (word length, text length, capitalization, and punctuation) or on lists of words likely to cause offense. Machine learning approaches (supervised, semi-supervised, or based on neural networks) rely on large annotated datasets, but many studies have shown that such approaches often fail because negativity in language may be wrapped in positive words, through metaphors and other figures of speech. Research, including our own, has found that accurately identifying and filtering toxic content requires a multidisciplinary perspective, drawing on a deep understanding of linguistics and on current methods in NLP and machine learning. To address existing gaps in the automatic detection of toxic comments, in the next five years I plan to: (Objective 1) study how metaphors and other figures of speech well known since antiquity (euphemism, litotes, hyperbole, sarcasm) convey toxic language. I will then develop (Objective 2) a system to detect figures of speech automatically, which I will integrate into (Objective 3) a new content moderation platform. The results of this work will mobilize research among scholars interested in evaluative language and the role of media in public discourse, including linguists, computational linguists, and communication and media researchers. At a time when media organizations, social media platforms, and the public are concerned about online abuse, misinformation, and the role of digital technology in politics and society, this project is timely and will make an important contribution to public discourse.