融合多模态信息的社交网络谣言检测方法

随着社交网络平台的发展,社交网络已经成为人们获取信息的重要来源。然而社交网络的便利性也导致了虚假谣言的快速传播。与纯文本的谣言相比,带有多媒体信息的网络谣言更容易误导用户以及被传播,因此对多模态的网络谣言检测在现实生活中有着重要意义。研究者们已提出若干多模态的网络谣言检测方法,但这些方法都没有充分挖掘出视觉特征和融合文本与视觉的联合表征特征。为弥补这些不足,提出了一个基于深度学习的端到端的多模态融合网络。该网络首先抽取出图片中各个兴趣区域的视觉特征,然后使用多头注意力机制将文本和视觉特征进行更新与融合,最后将这些特征进行基于注意力机制的拼接以用于社交网络多模态谣言检测。在推特和微博公开数据集上进行对比实验,结果表明,所提方法在推特数据集上F1值有13.4%的提升,在微博数据集上F1值有1.6%的提升。

Social Rumor Detection Method Based on Multimodal Fusion

With the development of social networking platforms,social networks have become an important source of information for people.However,the convenience of social networks has also led to the rapid propagation of false rumors.Compared with textual rumors,social network rumors with multimedia content are more likely to mislead users and get dissemination,so the detection of multi-modal rumors is of great significance in real life.Several multi-modal rumor detection methods have been proposed,but the visual features and joint representation of text and visual features have not been fully explored in current approaches.To make up for these shortcomings,an end-to-end multi-modal fusion network based on deep learning is developed.Firstly,the visual features of each region of interest in the image are extracted.Then,the text and the visual features are updated and fused by using a multi-head attention mechanism.Finally,these features are concatenated based on the attention mechanism for the detection of multi-modal rumors in social networks.Comparative experiments on the public data sets of Twitter and Weibo are conducted and experimental results show that the proposed method has a 13.4%F1 value increase on Twitter data set and a 1.6%F1 value increase on Weibo data set.