Online Hate Interpretation Varies by Country, But More by Individual: A Statistical Analysis Using Crowdsourced Ratings
Online Hate Interpretation Varies by Country, But More by Individual: A Statistical Analysis Using Crowdsourced Ratings
复制标题
在线仇恨解释因国家/地区而异，但因个人而异：使用众包评级的统计分析
DOI：
10.1109/snams.2018.8554954
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
B. Jansen
中科院分区：
文献类型：
--
作者：
Joni O. Salminen;Fabio Veronesi;Hind Almerekhi;Soon;B. Jansen
仇恨在网络社交媒体中十分普遍。这导致了大量关于检测和评分仇恨的研究。大多数计算工作涉及以众包评分作为训练数据的机器学习。这方面的一个突出例子是Perspective API，这是谷歌用于对网络评论的毒性进行评分的工具。然而，现有方法的一个主要问题是没有考虑网络仇恨的主观性。虽然有研究表明仇恨的强度各不相同且仇恨取决于语境，但没有研究系统地调查不同国家或个人对仇恨的解读有何差异。在这项探索性研究中，我们接受了这一挑战。我们从50个国家抽取众包工作者，让他们对相同的社交媒体评论的毒性进行评分，然后评估评分的差异，总共有18125个评分。我们发现不同国家之间的解读评分差异非常显著。然而，个人评分者之间对仇恨的解读差异比国家之间的差异更大。这些发现表明，仇恨评分系统在对网络仇恨进行评分和自动化处理时应该考虑用户层面的特征。
Hate is prevalent in online social media. This has resulted in a considerable amount of research in detecting and scoring it. Most computational efforts involve machine learning with crowdsourced ratings as training data. A prominent example of this is the Perspective API., a tool by Google to score toxicity of online comments. However., a major issue in the existing approaches is the lack of consideration for the subjective nature of online hate. While there is research that shows the intensity of hate varies and the hate depends on the context., there is no research that systematically investigates how hate interpretation varies by country or individual. In this exploratory research, we undertake this challenge. We sample crowd workers from 50 countries, have them score the same social media comments for toxicity and then evaluate the differences in the scores., altogether 18.,125 ratings. We find that the interpretation score differences among countries are highly significant. However., the hate interpretations vary more by the individual raters than by countries. These findings suggest that hate scoring systems should consider user-level features when scoring and automating the processing of online hate.