Multimedia venue semantic modeling based on multimodal data |
| |
| Affiliation: | 1. School of Electronic Information Engineering, Tianjin University, Tianjin 300072, China;2. School of Computing, National University of Singapore, Singapore;1. College of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361024, China;2. Department of Information Management, Chaoyang University of Technology, Taichung, Taiwan;1. Graduate School at Shenzhen, Tsinghua University, Shenzhen 518055, Guangdong, China;2. University of Virginia, Department of ECE, Charlottesville, VA 22904, USA;3. BJUT Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China;1. School of Computer Science and Electronic Engineering, Hunan University, 410082 Changsha, China;2. College of Computer and Communication Engineering, Changsha University of Science and Technology, 410112 Changsha, China |
| |
| Abstract: | A huge amount of text and multimedia (images and videos) data concerning venues is constantly being generated. To model the semantics of these venues, it is essential to analyze both text and multimedia user-generated content (UGC) in an integral manner. This task, however, is difficult for location-based social networks (LBSNs) because their text and multimedia UGCs tend to be uncorrelated. In this paper, we propose a novel multimedia location topic modeling approach to address this problem. We first utilize Recurrent Convolutional Networks to build the correlation between multimedia UGCs and text. Then, a graph model is structured according to these correlations. Next, we employ a graph clustering method to detect the latent multimedia topics for each venue. Based on the obtained venue semantics, we propose techniques to model multimedia location topics and perform semantic-based location summarization, venue prediction and image description. Extensive experiments are conducted on a cross-platform dataset, and the promising results demonstrate the superiority of the proposed method. |
| |
| Keywords: | Location-based Multimedia modeling Graph clustering Image description |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
