基于快速低秩编码与局部约束的图像分类算法

针对快速低秩编码算法存在特征重建误差较大,以及特征间局部约束条件丢失的问题,提出一种强化局部约束的快速低秩编码算法。首先,使用聚类算法对图像中特征进行聚类,得到局部相似特征集合及其对应的聚类中心;其次,在视觉词典中采取K最近邻（KNN）策略查找聚类中心对应的K个视觉单词,并将其组成对应的视觉词典;最后,使用快速低秩编码算法获得局部相似特征集合对应的特征编码。改进算法在Scene-15和Caltech-101图像库上的分类准确率比快速低秩编码算法提高4%到8%,编码效率比稀疏编码算法提高5~6倍。实验结果表明,改进算法使得局部相似特征具有相似编码,从而更加准确地表达图像内容,能有效提高分类准确率及编码效率。     

Discriminative sparse neighbor coding

Sparse coding has received extensive attention in the literature of image classification. Traditional sparse coding strategies tend to approximate local features in terms of a linear combination of basis vectors, without considering feature neighboring relationships. In this scenario, similar instances in the feature space may result in totally different sparse codes. To address this shortcoming, we investigate how to develop new sparse representations which preserve feature similarities. We commence by establishing two modules to improve the discriminative ability of sparse representation. The first module selects discriminative features for each class, and the second module eliminates non-informative visual words. We then explore the distribution of similar features over the dominant basis vectors for each class. We incorporate the feature distribution into the objective function, spanning a class-specific low dimensional subspace for effective sparse coding. Extensive experiments on various image classification tasks validate that the proposed approach consistently outperforms several state-of-the-art methods.     

Image classification using label constrained sparse coding

Sparse coding has been widely used for feature encoding in recent years. However, the encoded parameters’ similarity is ignored with sparse coding. Besides, the label information from which class the local feature is extracted is also ignored. To solve this problem, in this paper, we propose a novel feature encoding method called label constrained sparse coding (LCSC) for visual representation. The visual similarities between local features are jointly considered with the corresponding label information of local features. This is achieved by combining the label constraints with the encoding of local features. In this way, we can ensure that similar local features with the same label are encoded with similar parameters. Local features with different labels are encoded with dissimilar parameters to increase the discriminative power of encoded parameters. Besides, instead of optimizing for the coding parameter of each local feature separately, we jointly encode the local features within one sub-region in the spatial pyramid way to combine the spatial and contextual information of local features. We apply this label constrained sparse coding technique for classification tasks on several public image datasets to evaluate its effectiveness. The experimental results shows the effectiveness of the proposed method.     

构建近邻上下文的拷贝图像检索

目的 以词袋模型为基础的拷贝图像检索方法是当前最有效的方法。然而,由于局部特征量化存在信息损失,导致视觉词汇区别能力不足和视觉词汇误匹配增加,从而影响了拷贝图像检索效果。针对视觉词汇的误匹配问题,提出一种基于近邻上下文的拷贝图像检索方法。该方法通过局部特征的上下文关系消除视觉词汇歧义,提高视觉词汇的区分度,进而提高拷贝图像的检索效果。方法 首先,以距离和尺度关系选择图像中某局部特征点周围的特征点作为该特征点的上下文,选取的上下文中的局部特征点称为近邻特征点;再以近邻特征点的信息以及与该局部特征的关系为该局部特征构建上下文描述子;然后,通过计算上下文描述子的相似性对局部特征匹配对进行验证;最后,以正确匹配特征点的个数衡量图像间的相似性,并以此相似性选取若干候选图像作为返回结果。结果 在Copydays图像库进行实验,与Baseline方法进行比较。在干扰图像规模为100 k时,相对于Baseline方法,mAP提高了63%。当干扰图像规模从100 k增加到1 M时,Baseline的mAP值下降9%,而本文方法下降3%。结论 本文拷贝图像检索方法对图像编辑操作,如旋转、图像叠加、尺度变换以及裁剪有较高的鲁棒性。该方法可以有效地应用到图像防伪、图像去重等领域。     

Visual feature coding based on heterogeneous structure fusion for image classification

The relationship between visual words and local feature (words structure) or the distribution among images (images structure) is important in feature encoding to approximate the intrinsically discriminative structure of images in the Bag-of-Words (BoW) model. However, in recently most methods, the intrinsic invariance in intra-class images is difficultly captured using words structure or images structure for large variability image classification. To overcome this limitation, we propose a local visual feature coding based on heterogeneous structure fusion (LVFC-HSF) that explores the nonlinear relationship between words structure and images structure in feature space, as follows. First, we utilize high-order topology to describe the dependence of the visual words, and use the distance measurement based on the local feature to represent the distribution of images. Then, we construct the unitedly optimal framework according to the relevance between words structure and images structure to solve the projection matrix of local feature and the weight coefficient, which can exploit the nonlinear relationship of heterogeneous structure to balance their interaction. Finally, we adopt the improving fisher kernel(IFK) to fit the distribution of the projected features for obtaining the image feature. The experimental results on ORL, 15 Scenes, Caltech 101 and Caltech 256 demonstrate that heterogeneous structure fusion significantly enhances the intrinsic structure construction, and consequently improves the classification performance in these data sets.     

基于深度学习中间层卷积特征的图像标注

针对基于深度特征的图像标注模型训练复杂、时空开销大的不足,提出一种由深 度学习中间层特征表示图像视觉特征、由正例样本均值向量表示语义概念的图像标注方法。首 先,通过预训练深度学习模型的中间层直接输出卷积结果作为低层视觉特征,并采用稀疏编码 方式表示图像;然后,采用正例均值向量法为每个文本词汇构造视觉特征向量,从而构造出文 本词汇的视觉特征向量库;最后,计算测试图像与所有文本词汇的视觉特征向量相似度,并取 相似度最大的若干词汇作为标注词。多个数据集上的实验证明了所提出方法的有效性,就 F1 值而言,该方法在 IAPR TC-12 数据集上的标注性能比采用端到端深度特征的 2PKNN 和 JEC 分 别提高 32%和 60%。     

Tagging image by merging multiple features in a integrated manner

Image tagging is a task that automatically assigns the query image with semantic keywords called tags, which significantly facilitates image search and organization. Since tags and image visual content are represented in different feature space, how to merge the multiple features by their correlation to tag the query image is an important problem. However, most of existing approaches merge the features by using a relatively simple mechanism rather than fully exploiting the correlations between different features. In this paper, we propose a new approach to fusing different features and their correlation simultaneously for image tagging. Specifically, we employ a Feature Correlation Graph to capture the correlations between different features in an integrated manner, which take features as nodes and their correlations as edges. Then, a revised probabilistic model based on Markov Random Field is used to describe the graph for evaluating tag??s relevance to query image. Based on that, we design an image tagging algorithm for large scale web image dataset. We evaluate our approach using two large real-life corpuses collected from Flickr, and the experimental results indicate the superiority of our proposed approach over state-of-the-art techniques.     

哈希编码结合空间金字塔的图像分类

目的 稀疏编码是当前广泛使用的一种图像表示方法,针对稀疏编码及其改进算法计算过程复杂、费时等问题,提出一种哈希编码结合空间金字塔的图像分类算法。方法 首先,提取图像的局部特征点,构成局部特征点描述集。其次,学习自编码哈希函数,将局部特征点表示为二进制哈希编码。然后,在二进制哈希编码的基础上进行K均值聚类生成二进制视觉词典。最后,结合空间金字塔模型,将图像表示为空间金字塔直方图向量,并应用于图像分类。结果 在常用的Caltech-101和Scene-15数据集上进行实验验证,并和目前与稀疏编码相关的算法进行实验对比。与稀疏编码相关的算法相比,本文算法词典学习时间缩短了50%,在线编码速度提高了1.3～12.4倍,分类正确率提高了1%～5%。结论 提出了一种哈希编码结合空间金字塔的图像分类算法,利用哈希编码代替稀疏编码对局部特征点进行编码,并结合空间金字塔模型用于图像分类。实验结果表明,本文算法词典学习时间更短、编码速度更快,适用于在线词典学习和应用。     

Encoding Spatial Context for Large-Scale Partial-Duplicate Web Image Retrieval

Many recent state-of-the-art image retrieval approaches are based on Bag-of-Visual-Words model and represent an image with a set of visual words by quantizing local SIFT (scale invariant feature transform) features. Feature quantization reduces the discriminative power of local features and unavoidably causes many false local matches between images, which degrades the retrieval accuracy. To filter those false matches, geometric context among visual words has been popularly explored for the verification of geometric consistency. However, existing studies with global or local geometric verification are either computationally expensive or achieve limited accuracy. To address this issue, in this paper, we focus on partial duplicate Web image retrieval, and propose a scheme to encode the spatial context for visual matching verification. An efficient affine enhancement scheme is proposed to refine the verification results. Experiments on partial-duplicate Web image search, using a database of one million images, demonstrate the effectiveness and efficiency of the proposed approach. Evaluation on a 10-million image database further reveals the scalability of our approach.     

基于词包和特征融合的目标识别算法

针对现有词包模型对目标识别性能的不足,对特征提取、图像表示等方面进行改进以提高目标识别的准确率。首先,以密集提取关键点的方式取代SIFT关键点提取,减少了计算时间并最大程度地描述了图像底层信息。然后采用尺度不变特征变换(Scale-invariant feature transform, SIFT)描述符和统一模式的局部二值模式(Local binary pattern,LBP)描述符描述关键点周围的形状特征和纹理特征,引入K-Means聚类算法分别生成视觉词典,然后将局部描述符进行近似局部约束线性编码,并进行最大值特征汇聚。分别采用空间金字塔匹配生成具有空间信息的直方图,最后将金字塔直方图相串联,形成特征的图像级融合,并送入SVM进行分类识别。在公共数据库中进行实验,实验结果表明,本文所提方法能取得较高的目标识别准确率。     

Feature fusion within local region using localized maximum-margin learning for scene categorization

In the field of visual recognition such as scene categorization, representing an image based on the local feature (e.g., the bag-of-visual-word (BOVW) model and the bag-of-contextual-visual-word (BOCVW) model) has become popular and one of the most successful methods. In this paper, we propose a method that uses localized maximum-margin learning to fuse different types of features during the BOCVW modeling for eventual scene classification. The proposed method fuses multiple features at the stage when the best contextual visual word is selected to represent a local region (hard assignment) or the probabilities of the candidate contextual visual words used to represent the unknown region are estimated (soft assignment). The merits of the proposed method are that (1) errors caused by the ambiguity of single feature when assigning local regions to the contextual visual words can be corrected or the probabilities of the candidate contextual visual words used to represent the region can be estimated more accurately; and that (2) it offers a more flexible way in fusing these features through determining the similarity-metric locally by localized maximum-margin learning. The proposed method has been evaluated experimentally and the results indicate its effectiveness.     

一种视觉词软直方图的图像表示方法

基于视觉词的统计建模和判别学习,提出一种视觉词软直方图的图像表示方法.假设属于同一视觉词的图像局部特征服从高斯混合分布,利用最大-最小后验伪概率判别学习方法从样本中估计该分布,计算局部特征与视觉词的相似度.累加图像中每个视觉词与对应局部特征的相似度,在全部视觉词集合上进行结果的归一化,得到图像的视觉词软直方图.讨论了两种具体实现方法:一种是基于分类的软直方图方法,该方法根据相似度最大原则建立局部特征与视觉词的对应关系;另一种是完全软直方图方法,该方法将每个局部特征匹配到所有视觉词.在数据库Caltech-4和PASCAL VOC 2006上的实验结果表明,该方法是有效的.     

场景语义树图像标注方法

自动图像标注是一项具有挑战性的工作,它对于图像分析理解和图像检索都有着重要的意义.在自动图像标注领域,通过对已标注图像集的学习,建立语义概念空间与视觉特征空间之间的关系模型,并用这个模型对未标注的图像集进行标注.由于低高级语义之间错综复杂的对应关系,使目前自动图像标注的精度仍然较低.而在场景约束条件下可以简化标注与视觉特征之间的映射关系,提高自动标注的可靠性.因此提出一种基于场景语义树的图像标注方法.首先对用于学习的标注图像进行自动的语义场景聚类,对每个场景语义类别生成视觉场景空间,然后对每个场景空间建立相应的语义树.对待标注图像,确定其语义类别后,通过相应的场景语义树,获得图像的最终标注.在Corel5K图像集上,获得了优于TM(translation model)、CMRM(cross media relevance model)、CRM(continous-space relevance model)、PLSA-GMM(概率潜在语义分析-高期混合模型)等模型的标注结果.     

基于多尺度HOG的草图检索

草图检索是图像处理领域中的重要研究内容。提出了一种将高斯金字塔和局部HOG特征融合的特征提取改进方法,并将其用于草图检索。采用高斯金字塔将图像分解到多尺度空间,在所有尺度上进行兴趣点提取,获得基于兴趣点的多尺度HOG特征。利用图像的多尺度HOG特征集生成视觉词典,最终形成与视觉词典相关的特征描述向量,通过相似度匹配实现草图检索。将该算法与单一尺度下的HOG算法及其他几种算法比较,实验结果表明了其可行性和有效性。     

Using Language to Learn Structured Appearance Models for Image Annotation

Given an unstructured collection of captioned images of cluttered scenes featuring a variety of objects, our goal is to simultaneously learn the names and appearances of the objects. Only a small fraction of local features within any given image are associated with a particular caption word, and captions may contain irrelevant words not associated with any image object. We propose a novel algorithm that uses the repetition of feature neighborhoods across training images and a measure of correspondence with caption words to learn meaningful feature configurations (representing named objects). We also introduce a graph-based appearance model that captures some of the structure of an object by encoding the spatial relationships among the local visual features. In an iterative procedure, we use language (the words) to drive a perceptual grouping process that assembles an appearance model for a named object. Results of applying our method to three data sets in a variety of conditions demonstrate that, from complex, cluttered, real-world scenes with noisy captions, we can learn both the names and appearances of objects, resulting in a set of models invariant to translation, scale, orientation, occlusion, and minor changes in viewpoint or articulation. These named models, in turn, are used to automatically annotate new, uncaptioned images, thereby facilitating keyword-based image retrieval.