Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach
Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach
复制标题
DOI：
10.1145/3459637.3481908
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Chuanbo Hu;Minglei Yin;Bing Liu;Xin Li;Yanfang Ye
中科院分区：
文献类型：
--
作者：
Chuanbo Hu;Minglei Yin;Bing Liu;Xin Li;Yanfang Ye
Instagram和Twitter等社交媒体已成为营销和销售非法药物的重要平台。侦查网上非法药物贩运已成为打击网上非法药物贸易的关键。然而，法律的地位往往在空间和时间上有所不同;即使是同一种药物，联邦和州立法对其合法性也可能有不同的规定。与此同时，更多的贩毒事件被伪装成一种新型的广告-评论形式，导致信息异质性。因此，从社交媒体准确检测非法贩毒事件（IDTE）变得更具挑战性。在这项工作中，我们进行了第一次系统的研究，在Instagram上的IDTE的细粒度检测。我们建议采用深度多模态多标签学习（DMML）方法来检测IDTE，并在新构建的多模态IDTE（MM-IDTE）数据集上证明其有效性。具体来说，我们的模型以文本和图像数据为输入，并结合多模态信息来预测非法药物的多个标签。受BERT成功的启发，我们通过联合微调预训练的文本和图像编码器开发了一个自监督的多模态双向Transformer。我们已经构建了一个大规模的数据集MM-IDTE与手动注释的多个药物标签，以支持细粒度的非法药物检测。在MM-IDTE数据集上的大量实验结果表明，所提出的DMML方法可以准确地检测IDTE，即使在存在特殊字符和风格变化试图逃避检测。
Social media such as Instagram and Twitter have become important platforms for marketing and selling illicit drugs. Detection of online illicit drug trafficking has become critical to combat the online trade of illicit drugs. However, the legal status often varies spatially and temporally; even for the same drug, federal and state legislation can have different regulations about its legality. Meanwhile, more drug trafficking events are disguised as a novel form of advertising - commenting leading to information heterogeneity. Accordingly, accurate detection of illicit drug trafficking events (IDTEs) from social media has become even more challenging. In this work, we conduct the first systematic study on fine-grained detection of IDTEs on Instagram. We propose to take a deep multimodal multilabel learning (DMML) approach to detect IDTEs and demonstrate its effectiveness on a newly constructed dataset called multimodal IDTE (MM-IDTE). Specifically, our model takes text and image data as the input and combines multimodal information to predict multiple labels of illicit drugs. Inspired by the success of BERT, we have developed a self-supervised multimodal bidirectional transformer by jointly fine-tuning pretrained text and image encoders. We have constructed a large-scale dataset MM-IDTE with manually annotated multiple drug labels to support fine-grained detection of illicit drugs. Extensive experimental results on the MM-IDTE dataset show that the proposed DMML methodology can accurately detect IDTEs even in the presence of special characters and style changes attempting to evade detection.