Cluster-driven refinement for content-based digital image retrieval

Increasing application demands are pushing databases toward providing effective and efficient support for content-based retrieval over multimedia objects. In addition to adequate retrieval techniques, it is also important to enable some form of adaptation to users' specific needs. This paper introduces a new refinement method for retrieval based on the learning of the users' specific preferences. The proposed system indexes objects based on shape and groups them into a set of clusters, with each cluster represented by a prototype. Clustering constructs a taxonomy of objects by forming groups of closely-related objects. The proposed approach to learn the users' preferences is to refine corresponding clusters from objects provided by the users in the foreground, and to simultaneously adapt the database index in the background. Queries can be performed based solely on shape, or on a combination of shape with other features such as color. Our experimental results show that the system successfully adapts queries into databases with only a small amount of feedback from the users. The quality of the returned results is superior to that of a color-based query, and continues to improve with further use.     

Multiple-instance content-based image retrieval employing isometric embedded similarity measure

In image-based retrieval, global or local features sufficiently discriminative to summarize the image content are commonly extracted first. Traditional features, such as color, texture, shape or corner, characterizing image content are not reliable in terms of similarity measure. A good match in the feature domain does not necessarily map to image pairs with similar relationship. Applying these features as search keys may retrieve dissimilar false-positive images, or leave similar false-negative ones behind. Moreover, images are inherently ambiguous since they contain a great amount of information that justifies many different facets of interpretation. Using a single image to query a database might employ features that do not match user's expectation and retrieve results with low precision/recall ratios. How to automatically extract reliable image features as a query key that matches user's expectation in a content-based image retrieval (CBIR) system is an important topic.The objective of the present work is to propose a multiple-instance learning image retrieval system by incorporating an isometric embedded similarity measure. Multiple-instance learning is a way of modeling ambiguity in supervised learning given multiple examples. From a small collection of positive and negative example images, semantically relevant concepts can be derived automatically and employed to retrieve images from an image database. Each positive and negative example images are represented by a linear combination of fractal orthonormal basis vectors. The mapping coefficients of an image projected onto each orthonormal basis constitute a feature vector. The Euclidean-distance similarity measure is proved to remain consistent, i.e., isometric embedded, between any image pairs before and after the projection onto orthonormal axes. Not only similar images generate points close to each other in the feature space, but also dissimilar ones produce feature points far apart.The utilization of an isometric-embedded fractal-based technique to extract reliable image features, combined with a multiple-instance learning paradigm to derive relevant concepts, can produce desirable retrieval results that better match user's expectation. In order to demonstrate the feasibility of the proposed approach, two sets of test for querying an image database are performed, namely, the fractal-based feature extraction algorithm vs. three other feature extractors, and single-instance vs. multiple-instance learning. Both the retrieval results, execution time and precision/recall curves show favorably for the proposed multiple-instance fractal-based approach.     

On stability of signature-based similarity measures for content-based image retrieval

Retrieving similar images from large image databases is a challenging task for today’s content-based retrieval systems. Aiming at high retrieval performance, these systems frequently capture the user’s notion of similarity through expressive image models and adaptive similarity measures. On the query side, image models can significantly differ in quality compared to those stored on the database side. Thus, similarity measures have to be robust against these individual quality changes in order to maintain high retrieval performance. In this paper, we investigate the robustness of the family of signature-based similarity measures in the context of content-based image retrieval. To this end, we introduce the generic concept of average precision stability, which measures the stability of a similarity measure with respect to changes in quality between the query and database side. In addition to the mathematical definition of average precision stability, we include a performance evaluation of the major signature-based similarity measures focusing on their stability with respect to querying image databases by examples of varying quality. Our performance evaluation on recent benchmark image databases reveals that the highest retrieval performance does not necessarily coincide with the highest stability.     

互信息启发的相似度组合图像检索算法

图像的视觉特征与用户描述之间的差距一直是影响基于内容的图像检索准确度的最主要因素。对多种相似度进行组合来检索图像是近几年图像检索领域涌现出的一个研究热点,也是缩小这种差距的一种有效途径。如何选择更好的组合方法则是该领域很多研究者关注的核心问题。提出一种新的相似度组合算法。该算法基于互信息度量相对熵的原理,计算连续变量相似度与离散变量相似性之间的相关性,对多种相似度进行选择,以“和规则”组合相似度。在公用数据集上进行检索实验,该算法优于当前其他的“和规则”下的组合方法。     

Probabilistic relevance feedback approach for content-based image retrieval based on gaussian mixture models

A new relevance feedback (RF) approach for content-based image retrieval is presented. This approach uses Gaussian mixture (GM) models of the image features and a query that is updated in a probabilistic manner. This update reflects the preferences of the user and is based on the models of both the positive and negative feedback images. The retrieval is based on a recently proposed distance measure between probability density functions, which can be computed in closed form for GM models. The proposed approach takes advantage of the form of this distance measure and updates it very efficiently based on the models of the userspecified relevant and irrelevant images. It is also shown that this RF framework is fairly general and can be applied in case other image models or distance measures are used instead of those proposed in this work. Finally, comparative numerical experiments are provided, which that demonstrate the merits of the proposed RF methodology and the use of the distance measure, and also the advantages of using GMs for image modelling.     

Localized content-based image retrieval

We define localized content-based image retrieval as a CBIR task where the user is only interested in a portion of the image, and the rest of the image is irrelevant. In this paper we present a localized CBIR system, Accio, that uses labeled images in conjunction with a multiple-instance learning algorithm to first identify the desired object and weight the features accordingly, and then to rank images in the database using a similarity measure that is based upon only the relevant portions of the image. A challenge for localized CBIR is how to represent the image to capture the content. We present and compare two novel image representations, which extend traditional segmentation-based and salient point-based techniques respectively, to capture content in a localized CBIR setting.     

Learning similarity matching in multimedia content-based retrieval

Many multimedia content-based retrieval systems allow query formulation with the user setting the relative importance of features (e.g., color, texture, shape, etc.) to mimic the user's perception of similarity. However, the systems do not modify their similarity matching functions, which are defined during the system development. We present a neural network-based learning algorithm for adapting the similarity matching function toward the user's query preference based on his/her relevance feedback. The relevance feedback is given as ranking errors (misranks) between the retrieved and desired lists of multimedia objects. The algorithm is demonstrated for facial image retrieval using the NIST Mugshot Identification Database with encouraging results     

Adaptive tree similarity learning for image retrieval

Learning-enhanced relevance feedback is one of the most promising and active research directions in content-based image retrieval in recent years. However, the existing approaches either require prior knowledge of the data or converge slowly and are thus not coneffective. Motivated by the successful history of optimal adaptive filters, we present a new approach to interactive image retrieval based on an adaptive tree similarity model to solve these difficulties. The proposed tree model is a hierarchical nonlinear Boolean representation of a user query concept. Each path of the tree is a clustering pattern of the feedback samples, which is small enough and local in the feature space that it can be approximated by a linear model nicely. Because of the linearity, the parameters of the similartiy model are better learned by the optimal adaptive filter, which does not require any prior knowledge of the data and supports incremental learning with a fast convergence rate. The proposed approach is simple to implement and achieves better performance than most approaches. To illustrate the performance of the proposed approach, extensive experiments have been carried out on a large heterogeneous image collection with 17,000 images, which render promising results on a wide variety of queries.An early version of part of the system was reported in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition 2001.     

A Center-Surround Histogram for content-based image retrieval

In this paper, a new type of histogram which incorporates only the visual information surrounding the edges of the image is introduced. The edge extraction operation is performed with the use of a center-surround operator of the Human Visual System. The proposed Center-Surround Histogram (CSH) has two main advantages over the classic histogram. First, it reduces the amount of visual information that needs to be processed and second, it incorporates a degree of spatial information when used in content based image retrieval applications. The method is compared with other contemporary image retrieval methods, including that of another edge color histogram, on two different databases. The comparison shows that the use of CSH exhibits better results in shorter execution times.     

A multi-agent platform for content-based image retrieval

Efficient and possibly intelligent image retrieval is an important task, often required in many fields of human activity. While traditional database indexing techniques exhibit a remarkable performance in textual information retrieval current research in content-based image retrieval is focused on developing novel techniques that are biologically motivated and efficient. It is well known that humans have a remarkable ability to process visual information and to handle the volume and complexity of such information quite efficiently. In this paper, we present a content-based image retrieval platform that is based on a multi-agent architecture. Each agent is responsible for assessing the similarity of the query image to each candidate image contained in a collection based on a specific primitive feature and a corresponding similarity criterion. The outputs of various agents are integrated using one of several voting schemes supported by the system. The system’s performance has been evaluated using various collections of images, as well as images obtained in specific application domains such as medical imaging. The initial evaluation has yielded very promising results.

一种新的基于重排序的相关反馈图像检索方法

提出了一种新的相关反馈方法,该方法引入了R_norm重排序机制。通过计算用户反馈的按个人兴趣排列的期望输出顺序与系统输出图像顺序之间的R_norm值,来调整各个特征的权重,从而指导下一轮的检索。新方法不需标注,减轻了用户的负担,从而避免了用户是否愿意配合的问题,而且实验表明较Rui方法在性能上有很大提高。     

Probability based document clustering and image clustering using content-based image retrieval

Clustering of related or similar objects has long been regarded as a potentially useful contribution of helping users to navigate an information space such as a document collection. Many clustering algorithms and techniques have been developed and implemented but as the sizes of document collections have grown these techniques have not been scaled to large collections because of their computational overhead. To solve this problem, the proposed system concentrates on an interactive text clustering methodology, probability based topic oriented and semi-supervised document clustering. Recently, as web and various documents contain both text and large number of images, the proposed system concentrates on content-based image retrieval (CBIR) for image clustering to give additional effect to the document clustering approach. It suggests two kinds of indexing keys, major colour sets (MCS) and distribution block signature (DBS) to prune away the irrelevant images to given query image. Major colour sets are related with colour information while distribution block signatures are related with spatial information. After successively applying these filters to a large database, only small amount of high potential candidates that are somewhat similar to that of query image are identified. Then, the system uses quad modelling method (QM) to set the initial weight of two-dimensional cells in query image according to each major colour and retrieve more similar images through similarity association function associated with the weights. The proposed system evaluates the system efficiency by implementing and testing the clustering results with Dbscan and K-means clustering algorithms. Experiment shows that the proposed document clustering algorithm performs with an average efficiency of 94.4% for various document categories.     

Multiple representations, similarity matching, and results fusion for content-based image retrieval

In this paper, we show how the use of multiple content representations and their fusion can improve the performance of content-based image retrieval systems. We consider the case of texture and propose a new algorithm for texture retrieval based on multiple representations and their results fusion. Texture content is modeled using two different models: the well-known autoregressive model and a perceptual model based on perceptual features such as coarseness and directionality. In the case of the perceptual model, two viewpoints are considered: perceptual features are computed based on the original images viewpoint and on the autocovariance function viewpoint (corresponding to original images). So we consider a total of three content representations. The similarity measure used is based on Gower's index of similarity. Simple results of the fusion models are used to merge search results returned by different representations. Experimentations and benchmarking carried out on the well-known Brodatz database show a drastic improvement in search effectiveness with the fused model without necessarily altering their efficiency in an important way.     

A genetic programming framework for content-based image retrieval

The effectiveness of content-based image retrieval (CBIR) systems can be improved by combining image features or by weighting image similarities, as computed from multiple feature vectors. However, feature combination do not make sense always and the combined similarity function can be more complex than weight-based functions to better satisfy the users’ expectations. We address this problem by presenting a Genetic Programming framework to the design of combined similarity functions. Our method allows nonlinear combination of image similarities and is validated through several experiments, where the images are retrieved based on the shape of their objects. Experimental results demonstrate that the GP framework is suitable for the design of effective combinations functions.     

Complementary relevance feedback-based content-based image retrieval

We propose a complementary relevance feedback-based content-based image retrieval (CBIR) system. This system exploits the synergism between short-term and long-term learning techniques to improve the retrieval performance. Specifically, we construct an adaptive semantic repository in long-term learning to store retrieval patterns of historical query sessions. We then extract high-level semantic features from the semantic repository and seamlessly integrate low-level visual features and high-level semantic features in short-term learning to effectively represent the query in a single retrieval session. The high-level semantic features are dynamically updated based on users’ query concept and therefore represent the image’s semantic concept more accurately. Our extensive experimental results demonstrate that the proposed system outperforms its seven state-of-the-art peer systems in terms of retrieval precision and storage space on a large scale imagery database.