Relevance feedback in content-based image retrieval system by selective region growing in the feature space

This paper proposes a relevance feedback algorithm for the content-based image retrieval system. In the conventional algorithms, the weights of feature vectors are adjusted based on the user's feedback, which warps the match region from the hyper-sphere to hyper-ellipsoidal shape. That is, the axis grows into the direction that covers more relevant images in the feature space. The proposed algorithm is not based on the adjustment of the weights, but on the generation of new match region based on the user's feedback. Specifically, new spheres centered at the relevant images are generated, the radius of which are determined by the number of neighboring relevant and irrelevant images. The overall match region is the union of all the spheres generated and modified at each iteration of feedback process. As a result, the match region grows in a bubble shape into the direction where there are more relevant images. The resulting match region can cover arbitrarily shaped clusters whereas the weight updating approach can cover only hyper-ellipsoidal region. We also propose a data structure that keeps the history of past searches, for more rapid expansion of match region.     

Local tetra patterns: a new feature descriptor for content-based image retrieval

In this paper, we propose a novel image indexing and retrieval algorithm using local tetra patterns (LTrPs) for content-based image retrieval (CBIR). The standard local binary pattern (LBP) and local ternary pattern (LTP) encode the relationship between the referenced pixel and its surrounding neighbors by computing gray-level difference. The proposed method encodes the relationship between the referenced pixel and its neighbors, based on the directions that are calculated using the first-order derivatives in vertical and horizontal directions. In addition, we propose a generic strategy to compute nth-order LTrP using (n - 1)th-order horizontal and vertical derivatives for efficient CBIR and analyze the effectiveness of our proposed algorithm by combining it with the Gabor transform. The performance of the proposed method is compared with the LBP, the local derivative patterns, and the LTP based on the results obtained using benchmark image databases viz., Corel 1000 database (DB1), Brodatz texture database (DB2), and MIT VisTex database (DB3). Performance analysis shows that the proposed method improves the retrieval result from 70.34%/44.9% to 75.9%/48.7% in terms of average precision/average recall on database DB1, and from 79.97% to 85.30% and 82.23% to 90.02% in terms of average retrieval rate on databases DB2 and DB3, respectively, as compared with the standard LBP.     

基于显著性区域和蚁群算法的图像检索研究

针对提取到的图像特征受背景信息干扰,不能有针对性地提取到所需要的图像信息影响检索精度.为了解决这一问题,本文提出一种基于改进VGGNet(visual geometry group network)和蚁群算法的图像显著性区域检索算法.首先,利用类激活映射(class activation mapping,CMA)算法对...     

Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces, and progressive learning

Research has been devoted in the past few years to relevance feedback as an effective solution to improve performance of content-based image retrieval (CBIR). In this paper, we propose a new feedback approach with progressive learning capability combined with a novel method for the feature subspace extraction. The proposed approach is based on a Bayesian classifier and treats positive and negative feedback examples with different strategies. Positive examples are used to estimate a Gaussian distribution that represents the desired images for a given query; while the negative examples are used to modify the ranking of the retrieved candidates. In addition, feature subspace is extracted and updated during the feedback process using a principal component analysis (PCA) technique and based on user's feedback. That is, in addition to reducing the dimensionality of feature spaces, a proper subspace for each type of features is obtained in the feedback process to further improve the retrieval accuracy. Experiments demonstrate that the proposed method increases the retrieval speed, reduces the required memory and improves the retrieval accuracy significantly.     

基于视觉显著和CNN的花卉图像检索软件

花卉图像检索是图像检索领域的热门研究方向,高效、快速地检索数据库中的花卉图像一直是该方向的重点课题。为了检索花卉图像,文中设计了一个基于视觉显著模型和CNN的图像哈希算法,并根据此算法设计和开发出一个高效、快速的花卉图像检索软件。软件具有查询花卉类别、检索相似花卉、浏览花卉信息等功能。     

An occupancy model for image retrieval and similarity evaluation

The analysis of visual information often involves the manipulation of enormous volumes of data. If some tolerance is allowed in the results, orders of magnitude improvement in efficiency can be achieved in such analysis by appropriate selective processing, without necessarily considering all the data features. To guarantee that the error introduced does not exceed the allowed limit, a certain minimum proportion of the data must be involved in the analysis. This proportion cannot be determined arbitrarily. It should be chosen based on some formal methods, with a consideration of the error inherent in the data. This paper presents some techniques for improving the retrieval efficiency in image-based information systems, with performance guarantees on the reliability of results. Using the statistical theory of occupancy, it develops a model for the formal selection of the minimal subset of image features to be involved in histogram-based similarity evaluation. This guarantees that decisions based on the minimum proportion are always the same as (or close to) the one that would have been reached by considering all the features. Results on real and simulated data show the performance of the model on speedup, robustness, scalability, and performance guarantees.     

图像多角度局部特征提取及相似性匹配技术研究

图像特征提取是计算机视觉应用的根本基础.研究了SIFT、LBP和HOG等3种信息互补的局部特征(即多角度局部特征)提取算法,研究了基于稀疏编码的图像相似性匹配算法,并以基于内容的图像检索(CBIR)为应用实例,验证了算法的有效性和高效性.     

DRM: dynamic region matching for image retrieval using probabilistic fuzzy matching and boosting feature selection

This paper considers the semantic gap in content-based image retrieval from two aspects: (1) irrelevant visual contents (e.g. background) scatter the mapping from image to human perception; (2) unsupervised feature extraction and similarity ranking method can not accurately reveal users’ image perception. This paper proposes a novel region-based retrieval framework—dynamic region matching (DRM) to bridge the semantic gap. (1) To address the first issue, a probabilistic fuzzy region matching algorithm is adopted to retrieve and match images precisely at object level, which copes with the problem of inaccurate segmentation. (2) To address the second issue, a “FeatureBoost” algorithm is proposed to construct an effective “eigen” feature set in relevance feedback (RF) process. And the significance of each region is dynamically updated in RF learning to automatically capture users’ region of interest (ROI). (3) User’s retrieval purpose is predicted using a novel log-learning algorithm, which predicts users’ retrieval target in the feature space using the accumulated user operations. Extensive experiments have been conducted on Corel image database with over 10,000 images. The promising experimental results reveal the effectiveness of our scheme in bridging the semantic gap.     

基于SIFT特征的哈希快速检索与图像匹配

针对SIFT算法在应用于图像匹配时,存在准确率低下和耗时等问题,提出一种SIFT特征的哈希快速检索与图像匹配方法。文中提出以二值化SIFT关键点描述子和哈希表相结合的方法对图像进行匹配。针对实验过程中出现的冲突项,通过在哈希表中添加标志位并记录冲突相个数和地址,完美地解决了高维描述子转化到低维冲突项的问题,加快了匹配速度。实验结果表明,该方法图像匹配速度优于传统SIFT匹配方法,加快了相似特征检索速度、提高了查询效率,并能够满足实时应用。所提出的采用SIFT关键点描述子的二值化与哈希检索相结合的方法,通过对比实验,证明了该方法在保证准确率的同时,提高了效率,实现了图像的实时快速匹配。     

Structural image retrieval using automatic image annotation and region based inverted file

Image retrieval has lagged far behind text retrieval despite more than two decades of intensive research effort. Most of the research on image retrieval in the last two decades are on content based image retrieval or image retrieval based on low level features. Recent research in this area focuses on semantic image retrieval using automatic image annotation. Most semantic image retrieval techniques in literature, however, treat an image as a bag of features/words while ignore the structural or spatial information in the image. In this paper, we propose a structural image retrieval method based on automatic image annotation and region based inverted file. In the proposed system, regions in an image are treated the same way as keywords in a structural text document, semantic concepts are learnt from image data to label image regions as keywords and weight is assigned to each keyword according to spatial position and relationship. As the result, images are indexed and retrieved in the same way as structural document retrieval. Specifically, images are broken down to regions which are represented using colour, texture and shape features. Region features are then quantized to create visual dictionaries which are similar to monolingual dictionaries like English or Chinese dictionaries. In the next step, a semantic dictionary similar to a bilingual dictionary like the English–Chinese dictionary is learnt to mapping image regions to semantic concepts. Finally, images are then indexed and retrieved using a novel region based inverted file data structure. Results show the proposed method has significant advantage over the widely used Bayesian annotation models.     

特征结合和相关反馈技术在医学图像检索中的应用

本文在单一检索的基础上,对基于灰度直方图的颜色特征,基于小波变换的纹理特征和基于不变矩的形状特征进行融合.为了使用户能够参与检索过程,又引入了相关反馈机制,通过调整权值使得检索的结果最终满足用户的检索要求.最后分别给出基于单一特征,特征融合和相关反馈方法的查准率和查全率,并对试验结果进行分析.     

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.     

A unified image retrieval framework on local visual and semantic concept-based feature spaces

This paper presents a learning-based unified image retrieval framework to represent images in local visual and semantic concept-based feature spaces. In this framework, a visual concept vocabulary (codebook) is automatically constructed by utilizing self-organizing map (SOM) and statistical models are built for local semantic concepts using probabilistic multi-class support vector machine (SVM). Based on these constructions, the images are represented in correlation and spatial relationship-enhanced concept feature spaces by exploiting the topology preserving local neighborhood structure of the codebook, local concept correlation statistics, and spatial relationships in individual encoded images. Finally, the features are unified by a dynamically weighted linear combination of similarity matching scheme based on the relevance feedback information. The feature weights are calculated by considering both the precision and the rank order information of the top retrieved relevant images of each representation, which adapts itself to individual searches to produce effective results. The experimental results on a photographic database of natural scenes and a bio-medical database of different imaging modalities and body parts demonstrate the effectiveness of the proposed framework.     

基于小波纹理、语义特征和相关反馈的医学图像检索

本文针对胸片和胸部CT扫描两种医学图像库，提出一种融合小波纹理、语义特征和相关反馈的医学图像检索技术。首先分析比较了纹理的三大描述方法：灰度层共现矩阵，纹理谱，小波变换，采用了频谱法中的一种基于小波变换的纹理描述方法进行纹理分析。为了进一步提高检索精度，将语义特征融合到该算法中，然后利用相关反馈技术有效地提高检索准确率。通过在原型系统上的实验，从查准率、查全率、平均序号验证了此方法的有效性。     

基于区域加权信息熵和改进遗传算法的图像检索

采用信息熵性能评价指标从概率的角度描述因权值变化而引起的图像信息熵分布的变化,可以根据用户关注的范围确定重点加权区域,通过改进的猴王遗传算法来确定权值选择,能够自动的确定感兴趣的区域。区域加权信息熵能够提取图像的颜色以及空间特性,并且保留了香农熵的数学特性。实验表明,区域加权信息熵实验结果的准确率高于单纯的颜色特征提取算法。     

A new SVM-based relevance feedback image retrieval using probabilistic feature and weighted kernel function

Relevance feedback (RF) is an effective approach to bridge the gap between low-level visual features and high-level semantic meanings in content-based image retrieval (CBIR). The support vector machine (SVM) based RF mechanisms have been used in different fields of image retrieval, but they often treat all positive and negative feedback samples equally, which will inevitably degrade the effectiveness of SVM-based RF approaches for CBIR. In fact, positive and negative feedback samples, different positive feedback samples, and different negative feedback samples all always have distinct properties. Moreover, each feedback interaction process is usually tedious and time-consuming because of complex visual features, so if too many times of iteration of feedback are asked, users may be impatient to interact with the CBIR system. To overcome the above limitations, we propose a new SVM-based RF approach using probabilistic feature and weighted kernel function in this paper. Firstly, the probabilistic features of each image are extracted by using principal components analysis (PCA) and the adapted Gaussian mixture models (AGMM) based dimension reduction, and the similarity is computed by employing Kullback–Leibler divergence. Secondly, the positive feedback samples and negative feedback samples are marked, and all feedback samples’ weight values are computed by utilizing the samples-based Relief feature weighting. Finally, the SVM kernel function is modified dynamically according to the feedback samples’ weight values. Extensive simulations on large databases show that the proposed algorithm is significantly more effective than the state-of-the-art approaches.     

A novel unsupervised multiple feature hashing for image retrieval and indexing (MFHIRI)

Recently, techniques that can automatically figure out the incisive information from gigantic visual databases are urging popularity. The existing multi-feature hashing method has achieved good results by fusing multiple features, but in processing these multi-features, fusing multi-features into one feature will cause the feature dimension to be very high, increasing the amount of calculation. On the one hand, it is not easy to discover the internal ties between different features. This paper proposes a novel unsupervised multiple feature hashing for image retrieval and indexing (MFHIRI) method to learn multiple views in a composite manner. The proposed scheme learns the binary codes of various information sources in a composite manner, and our scheme relies on weighted multiple information sources and improved KNN concept. In particular, here we adopt an adaptive weighing scheme to preserve the similarity and consistency among binary codes. Precisely, we follow the graph modeling theory to construct improved KNN concept, which further helps preserve different statistical properties of individual sources. The important aspect of improved KNN scheme is that we can find the neighbors of a data point by searching its neighbors’ neighbors. During optimization, the sub-problems are solved in parallel which efficiently lowers down the computation cost. The proposed approach shows consistent performance over state-of-the-art (three single-view and eight multi-view approaches) on three broadly followed datasets viz. CIFAR-10, NUS-WIDE and Caltech-256.     

Content based image indexing and retrieval using directional local extrema and magnitude patterns

In this paper, we integrate the concept of directional local extremas and their magnitude based patterns for content based image indexing and retrieval. The standard ditectional local extrama pattern (DLEP) extracts the directional edge information based on local extrema in 0°, 45°, 90°, and 135° directions in an image. However, they are not considering the magnitudes of local extremas. The proposed method integrates these two concepts for better retrieval performance. The sign DLEP (SDLEP) operator is a generalized DLEP operator and magnitude DLEP (MDLEP) operator is calculated using magnitudes of local extremas. The performance of the proposed method is compared with DLEP, local binary patterns (LBPs), block-based LBP (BLK_LBP), center-symmetric local binary pattern (CS-LBP), local edge patterns for segmentation (LEPSEG) and local edge patterns for image retrieval (LEPINV) methods by conducting two experiments on benchmark databases, viz. Corel-5K and Corel-10K databases. The results after being investigated show a significant improvement in terms of their evaluation measures as compared to other existing methods on respective databases.     

Indexing and encoding based image feature representation with bin overlapped similarity measure for CBIR applications

In Content Based Image Retrieval (CBIR) system, the exhaustive search for a given query image to find the relevant images in the database are non-scalable. In this paper, we propose indexing, coding technique and similarity measure to address the above mentioned problem. We consider the color histogram of the image and its bin values are analyzed to understand the color information in the image. The histogram dimension is reduced by removing trivial bins and only those bins that represent color information significantly are considered. Based on the dimensions of the histogram, it is clustered and indexed. The Golomb–Rice (GR) coding is used to encode the indexed histograms. The Bin Overlapped Similarity Measure (BOSM) is proposed to compute the distance values between query and database image histograms. The performance of proposed approach is evaluated on benchmark datasets and found that the performance of the proposed approach is encouraging.