Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine

In recent years, the rapid growth of multimedia content makes content-based image retrieval (CBIR) a challenging research problem. The content-based attributes of the image are associated with the position of objects and regions within the image. The addition of image content-based attributes to image retrieval enhances its performance. In the last few years, the bag-of-visual-words (BoVW) based image representation model gained attention and significantly improved the efficiency and effectiveness of CBIR. In BoVW-based image representation model, an image is represented as an order-less histogram of visual words by ignoring the spatial attributes. In this paper, we present a novel image representation based on the weighted average of triangular histograms (WATH) of visual words. The proposed approach adds the image spatial contents to the inverted index of the BoVW model, reduces overfitting problem on larger sizes of the dictionary and semantic gap issues between high-level image semantic and low-level image features. The qualitative and quantitative analysis conducted on three image benchmarks demonstrates the effectiveness of the proposed approach based on WATH.     

基于感知哈希和视觉词袋模型的图像检索方法

针对移动增强现实中图像检索技术耗时长导致的实时性不高的问题,提出了一种 基于感知哈希和视觉词袋模型结合的图像检索方法。图像检索过程中,在保证一定正确率的基 础上加快了检索速度。首先,对数据集图像使用改进的感知哈希技术处理,选取与查询相似的 图像集合,达到筛选图像数据集的作用;然后,对相似图像集使用视觉词袋模型进行图像检索, 选取和查询图像中目标一致的目标图像。实验结果表明,该方法相比较视觉词袋模型算法检索 的平均正确率提高了 3.2%,检索时间缩短了 102.9 ms,能够满足移动增强现实中图像检索的实 时性要求,为移动增强现实系统提供了有利的条件。     

A robust CBIR framework in between bags of visual words and phrases models for specific image datasets

One objective of the Content Based Image Retrieval research field is to propose new methodologies and tools to manage the increasing number of images available. Linked to a specific context of small expert datasets without prior knowledge, our research work focuses on improving the discriminative power of the image representation while keeping the same efficiency for retrieval. Based on the well-known bag of visual words model, we propose three different methodologies inspired by the visual phrase model effectiveness and a compression technique which ensures the same effectiveness for retrieval than the BoVW model. Our experimental results study the performance of our proposals on different well known benchmark datasets and show its good performance compared to other recent approaches.     

一种基于视觉词典优化和查询扩展的图像检索方法

视觉词典方法（Bag of visual words,BoVW）是当前图像检索领域的主流方法,然而,传统的视觉词典方法存在计算量大、词典区分性不强以及抗干扰能力差等问题,难以适应大数据环境.针对这些问题,本文提出了一种基于视觉词典优化和查询扩展的图像检索方法.首先,利用基于密度的聚类方法对SIFT特征进行聚类生成视觉词典,提高视觉词典的生成效率和质量;然后,通过卡方模型分析视觉单词与图像目标的相关性,去除不包含目标信息的视觉单词,增强视觉词典的分辨能力;最后,采用基于图结构的查询扩展方法对初始检索结果进行重排序.在Oxford5K和Paris6K图像集上的实验结果表明,新方法在一定程度上提高了视觉词典的质量和语义分辨能力,性能优于当前主流方法.     

基于加权特征空间信息视觉词典的图像检索模型

针对传统的视觉词袋模型中视觉词典对底层特征量化时容易引入量化误差,以及视觉单词的适用性不足等问题,提出了基于加权特征空间信息视觉词典的图像检索模型。从产生视觉词典的常用聚类算法入手,分析和探讨了聚类算法的特点,考虑聚类过程中特征空间的特征分布统计信息,通过实验对不同的加权方式进行对比,得出效果较好的均值加权方案,据此对视觉单词的重要程度加权,提高视觉词典的描述能力。对比实验表明,在ImageNet图像数据集上,相对于同源视觉词典,非同源视觉词典对视觉空间的划分影响较小,且基于加权特征空间信息视觉词典在大数据集上更加有效。     

Visual word spatial arrangement for image retrieval and classification

We present word spatial arrangement (WSA), an approach to represent the spatial arrangement of visual words under the bag-of-visual-words model. It lies in a simple idea which encodes the relative position of visual words by splitting the image space into quadrants using each detected point as origin. WSA generates compact feature vectors and is flexible for being used for image retrieval and classification, for working with hard or soft assignment, requiring no pre/post processing for spatial verification. Experiments in the retrieval scenario show the superiority of WSA in relation to Spatial Pyramids. Experiments in the classification scenario show a reasonable compromise between those methods, with Spatial Pyramids generating larger feature vectors, while WSA provides adequate performance with much more compact features. As WSA encodes only the spatial information of visual words and not their frequency of occurrence, the results indicate the importance of such information for visual categorization.     

基于视觉词模糊权重的视频语义标注

针对视觉词袋模型的量化误差与视觉词含糊性,提出一种基于视觉词模糊权重的视频语义标注方案。该方案在训练样本集的预聚类基础上,逐个聚类训练单类支持向量机OC-SVM。根据样本特征与聚类超球球心的距离函数及聚类超球的空间分布确定视觉词映射及权重,以提高视觉词的表达力、区别力。实验结果表明,基于该方案的视频语义标注精度分别比TF方案和VWA方案提高34%和16%。     

一种视觉词软直方图的图像表示方法

基于视觉词的统计建模和判别学习,提出一种视觉词软直方图的图像表示方法.假设属于同一视觉词的图像局部特征服从高斯混合分布,利用最大-最小后验伪概率判别学习方法从样本中估计该分布,计算局部特征与视觉词的相似度.累加图像中每个视觉词与对应局部特征的相似度,在全部视觉词集合上进行结果的归一化,得到图像的视觉词软直方图.讨论了两种具体实现方法:一种是基于分类的软直方图方法,该方法根据相似度最大原则建立局部特征与视觉词的对应关系;另一种是完全软直方图方法,该方法将每个局部特征匹配到所有视觉词.在数据库Caltech-4和PASCAL VOC 2006上的实验结果表明,该方法是有效的.     

Constructing a discriminative visual vocabulary with macro and micro sense of visual words

Visual vocabulary representation approach has been successfully applied to many multimedia and vision applications, including visual recognition, image retrieval, and scene modeling/categorization. The idea behind the visual vocabulary representation is that an image can be represented by visual words, a collection of local features of images. In this work, we will develop a new scheme for the construction of visual vocabulary based on the analysis of visual word contents. By considering the content homogeneity of visual words, we design a visual vocabulary which contains macro-sense and micro-sense visual words. The two types of visual words are appropriately further combined to describe an image effectively. We also apply the visual vocabulary to construct image retrieving and categorization systems. The performance evaluation for the two systems indicates that the proposed visual vocabulary achieves promising results.     

用于视觉词语生成的概率预测器

视觉词语的产生是基于字袋模型的图像检索中的重要一环:根据已知的视觉词典,查询图像特征被映射到词典中相应的视觉词语。提出一种新的基于空间相关性的快速视觉词语产生算法。统计视觉词典中任意两个词语在数据库中的共生次数,构建视觉词语共生表。利用共生表,建立一种新的概率预测器来辅助预测已知词语的近邻词语。将预测器与快速近似最近邻查找算法结合,在标准图像检索数据库上进行实验测试,相比较传统的树形搜索算法或哈希算法,新算法在时间效率上获得明显提高。     

一种基于稀疏典型性相关分析的图像检索方法

图像语义检索的一个关键问题就是要找到图像底层特征与语义之间的关联,由于文本是表达语义的一种有效手段,因此提出通过研究文本与图像两种模态之间关系来构建反映两者间潜在语义关联的有效模型的思路,基于该模型,可使用自然语言形式(文本语句)来表达检索意图,最终检索到相关图像.该模型基于稀疏典型性相关分析(sparse canonical correlation analysis,简称sparse CCA),按照如下步骤训练得到:首先利用隐语义分析方法构造文本语义空间,然后以视觉词袋(bag of visual words)来表达文本所对应的图像,最后通过Sparse CCA算法找到一个语义相关空间,以实现文本语义与图像视觉单词间的映射.使用稀疏的相关性分析方法可以提高模型可解释性和保证检索结果稳定性.实验结果验证了Sparse CCA方法的有效性,同时也证实了所提出的图像语义检索方法的可行性.     

A comparative study on text representation schemes in text categorization

It is well known that the classification effectiveness of the text categorization system is not simply a matter of learning algorithms. Text representation factors are also at work. This paper will consider the ways in which the effectiveness of text classifiers is linked to the five text representation factors: “stop words removal”, “word stemming”, “indexing”, “weighting”, and “normalization”. Statistical analyses of experimental results show that performing “normalization” can always promote effectiveness of text classifiers significantly. The effects of the other factors are not as great as expected. Contradictory to common sense, a simple binary indexing method can sometimes be helpful for text categorization.     

Multiple visual concept discovery using concept-based visual word clustering

In recent research, visual concept discovery was used to fill the semantic gap for representing the visual content. However, multiple concepts in an image generally degrade the discovery accuracy. In this paper, a Concept-based Visual Word Clustering (CVWC) method is proposed to discover multiple concepts from an image without pre-segmented training images. The CVWC is based on prior knowledge of concepts, which are trained from meta-text of web images. First, concepts are obtained by clustering the visual words in the regions extracted from image segmentation. A concept-based genetic algorithm (CBGA) is applied for searching the near-optimal clusters according to the visual words (VWs) in a concept and the co-occurrence probability of two concepts. The clustering procedure is also performed on the neighboring VWs to discover all the regions for concept representation. A concept extension method (CE) is further applied for iteratively updating the discovered concepts from the clustered results. In the experiments on the application to video retrieval, the mAP of the proposed CVWC method based on CBGA and CE obtained satisfactory improvements of 0.04 and 0.06, compared to pixel-based image segmentation approach and conventional concept model approach for the category “nation defense,” and 0.06 and 0.05 for the category “ecology,” respectively.