首页 | 本学科首页   官方微博 | 高级检索  
     

基于文本引导的注意力图像转发预测排序网络
引用本文:潘文雯, 赵洲, 俞俊, 吴飞. 基于文本引导的注意力图像转发预测排序网络. 自动化学报, 2021, 47(11): 2547−2556 doi: 10.16383/j.aas.c200629
作者姓名:潘文雯  赵洲  俞俊  吴飞
作者单位:1.浙江大学计算机科学与技术学院 杭州 310027;;2.杭州电子科技大学计算机科学与技术学院 杭州 310018
基金项目:国家重点研发计划(2018AAA0100603), 浙江省自然基金(LR19F020006), 国家自然科学基金项目(61836002, U1611461, 61751209)资助
摘    要:转发预测在社交媒体网站(Social media sites, SMS)中是一个很有挑战性的问题. 本文研究了SMS中的图像转发预测问题, 预测用户再次转发图像推特的图像共享行为. 与现有的研究不同, 本文首先提出异构图像转发建模网络(Image retweet modeling, IRM), 所利用的是用户之前转发图像推特中的相关内容、之后在SMS中的联系和被转发者的偏好三方面的内容. 在此基础上, 提出文本引导的多模态神经网络, 构建新型多方面注意力排序网络学习框架, 从而学习预测任务中的联合图像推特表征和用户偏好表征. 在Twitter的大规模数据集上进行的大量实验表明, 我们的方法较之现有的解决方案而言取得了更好的效果.

关 键 词:图像转发预测   多模态学习   文本引导   注意力机制
收稿时间:2020-08-10
修稿时间:2020-10-15

Textually Guided Ranking Network for Attentional Image Retweet Modeling
Pan Wen-Wen, Zhao Zhou, Yu Jun, Wu Fei. Textually guided ranking network for attentional image retweet modeling. Acta Automatica Sinica, 2021, 47(11): 2547−2556 doi: 10.16383/j.aas.c200629
Authors:PAN Wen-Wen  ZHAO Zhou  YU Jun  WU Fei
Affiliation:1. School of Computer Science and Technology, Zhejiang University, Hangzhou 310027;;2. School of Computer and Science, Hangzhou Dianzi University, Hangzhou 310018
Abstract:Retweet prediction is a challenging problem in social media sites (SMS). In this paper, we study the problem of image retweet prediction in social media, which predicts the image sharing behavior that the user reposts the image tweets from their followees. Unlike previous studies, we learn user preference ranking model from their past retweeted image tweets in SMS. We first propose a heterogeneous image retweet modeling network (IRM) that exploits users past retweeted image tweets with associated contexts, their following relations in SMS and preference of their followees. We then develop a novel attentional multi-faceted ranking network learning framework with textually guided multi-modal neural networks for the proposed heterogenous IRM network to learn the joint image tweet representations and user preference representations for prediction task. The extensive experiments on a large-scale dataset from Twitter site show that our method achieves better performance than other state-of-the-art solutions to the problem.
Keywords:Image retweet prediction  multi-modal learning  textually guided prediction  attention mechanism
点击此处可从《自动化学报》浏览原始摘要信息
点击此处可从《自动化学报》下载全文
设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司  京ICP备09084417号