Hybrid textual-visual relevance learning for content-based image retrieval |
| |
| Affiliation: | 1. School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan 250014, China;2. School of Computer Science and Technology, Shandong University, Jinan 250101, China;1. School of Information Science and Engineering, Lanzhou University, Lanzhou, China;2. Faculty of Science and Technology, Bournemouth University, UK;1. School of Electronic Information Engineering, Tianjin University, Tianjin 300072, China;2. School of Computing, National University of Singapore, Singapore;1. Department of Computer Science, University of California, Irvine, USA;2. School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, China;3. Faculty of Information and Communication Technology, Mahidol University, Thailand;1. Beijing Key Lab of Intelligent Telecomm. Software and Multimedia, Beijing University of Posts and Telecomm., Beijing 100876, China;2. School of Information Technology, Deakin University, Melbourne, VIC 3125, Australia;3. Department of Computer Science and Technology, Tsinghua University, Beijing, China;1. School of Computer and Information, Hefei University of Technology, Hefei, China;2. School of Information Engineering, Wuhan University of Technology, Wuhan, China;3. Huazhong University of Science and Technology, Wuhan, China;1. School of Electrical Automation and Information Engineering, Tianjin University, Tianjin, PR China;2. Dept. of Computer Science, University College London, London WC1E 6EA, UK;3. National Key Laboratory of Science and Technology on Aerospace Intelligence Control, Beijing, PR China;4. Department of Computer Science, School of Science at Loughborough University, UK;5. Department of Information Engineering and Computer Science, University of Trento, Italy |
| |
| Abstract: | Learning effective relevance measures plays a crucial role in improving the performance of content-based image retrieval (CBIR) systems. Despite extensive research efforts for decades, how to discover and incorporate semantic information of images still poses a formidable challenge to real-world CBIR systems. In this paper, we propose a novel hybrid textual-visual relevance learning method, which mines textual relevance from image tags and combines textual relevance and visual relevance for CBIR. To alleviate the sparsity and unreliability of tags, we first perform tag completion to fill the missing tags as well as correct noisy tags of images. Then, we capture users’ semantic cognition to images by representing each image as a probability distribution over the permutations of tags. Finally, instead of early fusion, a ranking aggregation strategy is adopted to sew up textual relevance and visual relevance seamlessly. Extensive experiments on two benchmark datasets well verified the promise of our approach. |
| |
| Keywords: | Content-based image retrieval Tag completion Semantics modeling Rank aggregation Sparse linear method |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
