A Multitask Framework for Sentiment, Emotion and Sarcasm aware Cyberbullying Detection from Multi-modal Code-Mixed Memes
A Multitask Framework for Sentiment, Emotion and Sarcasm aware Cyberbullying Detection from Multi-modal Code-Mixed Memes
复制标题
用于从多模态代码混合模因中检测情感、情感和讽刺的网络欺凌的多任务框架
DOI：
10.1145/3477495.3531925
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
P. Bhattacharyya
中科院分区：
文献类型：
--
作者：
Krishanu Maity;Prince Jha;S. Saha;P. Bhattacharyya
从模因中检测网络欺凌具有很大的挑战性，因为存在隐含的情感内容，这些内容通常也是讽刺性的，并且是多模态的（图像+文本）。目前的工作是第一次尝试，尽我们所知，在调查的作用，情绪，情感和讽刺识别网络欺凌的多模态模因在代码混合的语言设置。作为贡献，我们创建了一个基准多模态模因数据集，称为MultiBully，并从开源Twitter和Reddit平台收集了欺凌，情感，情绪和讽刺标签。此外，网络欺凌帖子的严重程度也通过为每个模因添加危害性得分来调查。创建的数据集由两种模态组成，文本和图像。我们数据集中的大多数文本都是代码混合形式的，这为多语言用户捕获了语言之间的无缝转换。两种不同的多模态多任务框架（BERT+ResNET-Feedback和CLIP-CentralNet）已被提出用于网络欺凌检测（CD），三个辅助任务是情感分析（SA），情感识别（ER）和讽刺检测（SAR）。实验结果表明，与单模态和单任务变体相比，所提出的框架提高了主任务的性能，即，CD，准确率和F1评分分别提高3.18%和3.10%。
Detecting cyberbullying from memes is highly challenging, because of the presence of the implicit affective content which is also often sarcastic, and multi-modality (image + text). The current work is the first attempt, to the best of our knowledge, in investigating the role of sentiment, emotion and sarcasm in identifying cyberbullying from multi-modal memes in a code-mixed language setting. As a contribution, we have created a benchmark multi-modal meme dataset called MultiBully annotated with bully, sentiment, emotion and sarcasm labels collected from open-source Twitter and Reddit platforms. Moreover, the severity of the cyberbullying posts is also investigated by adding a harmfulness score to each of the memes. The created dataset consists of two modalities, text and image. Most of the texts in our dataset are in code-mixed form, which captures the seamless transitions between languages for multilingual users. Two different multimodal multitask frameworks (BERT+ResNET-Feedback and CLIP-CentralNet) have been proposed for cyberbullying detection (CD), the three auxiliary tasks being sentiment analysis (SA), emotion recognition (ER) and sarcasm detection (SAR). Experimental results indicate that compared to uni-modal and single-task variants, the proposed frameworks improve the performance of the main task, i.e., CD, by 3.18% and 3.10% in terms of accuracy and F1 score, respectively.