A dynamic weighted multimodal fusion fake information detection method
A dynamic weighted multimodal fusion fake information detection method
复制标题
DOI：
--
复制
发表时间：
2023
影响因子：
6.6
通讯作者：
Arun Kumar Sangaiah
中科院分区：
文献类型：
--
作者：
Lulan Zuo;Zhiyong Zhang;Jian Wang;Arun Kumar Sangaiah
Regarding the task of recognizing fake information, it is difficult to perform accurate recognition based on single-modal fake information detection models in the face of combined graphic and textual fake information. To address the problem wherein current multimodal detection models usually use splicing for multimodal fusion, which leads to a redundancy of modal information during feature fusion and cannot effectively combine the advantages of different modalities, this paper proposes a dynamic weighted multimodal fusion network for feature fusion based on the attention mechanism. To address the problem of inadequate text content extraction, extracting text abstract features using the TextRank algorithm, and the abstract features are introduced into the detection model as an independent modality. The dynamic weighted multimodal disinformation detection model (DWMF) proposed by Text uses BERT and Vit to extract text and image features, respectively, after which the text, image, and abstract features are fused using a fusion network and then classified. The model achieves an accuracy rate of 98.1% with the MCG-FNeWS public dataset, and it F1 score and accuracy are better than those of existing multimodal models.
