Visual information retrieval system via content-based approach

In this paper a content-based image retrieval method that can search large image databases efficiently by color, texture, and shape content is proposed. Quantized RGB histograms and the dominant triple (hue, saturation, and value), which are extracted from quantized HSV joint histogram in the local image region, are used for representing global/local color information in the image. Entropy and maximum entry from co-occurrence matrices are used for texture information and edge angle histogram is used for representing shape information. Relevance feedback approach, which has coupled proposed features, is used for obtaining better retrieval accuracy. A new indexing method that supports fast retrieval in large image databases is also presented. Tree structures constructed by k-means algorithm, along with the idea of triangle inequality, eliminate candidate images for similarity calculation between query image and each database image. We find that the proposed method reduces calculation up to average 92.2 percent of the images from direct comparison.     

Localized content-based image retrieval

We define localized content-based image retrieval as a CBIR task where the user is only interested in a portion of the image, and the rest of the image is irrelevant. In this paper we present a localized CBIR system, Accio, that uses labeled images in conjunction with a multiple-instance learning algorithm to first identify the desired object and weight the features accordingly, and then to rank images in the database using a similarity measure that is based upon only the relevant portions of the image. A challenge for localized CBIR is how to represent the image to capture the content. We present and compare two novel image representations, which extend traditional segmentation-based and salient point-based techniques respectively, to capture content in a localized CBIR setting.     

Association and content-based retrieval

In spite of important efforts in content-based indexing and retrieval during these last years, seeking relevant and accurate images remains a very difficult query. In the state-of-the-art approaches, the retrieval task may be efficient for some queries in which the semantic content of the query can be easily translated into visual features. For example, finding images of fires is simple because fires are characterized by specific colors (yellow and red). However, it is not efficient in other application fields in which the semantic content of the query is not easily translated into visual features. For example, finding images of birds during migrations is not easy because the system has to understand the query semantic. In the query, the basic visual features may be useful (a bird is characterized by a texture and a color), but they are not sufficient. What is missing is the generalization capability. Birds during migrations belong to the same repository of birds, so they share common associations among basic features (e.g., textures and colors) that the user cannot specify explicitly. We present an approach that discovers hidden associations among features during image indexing. These associations discriminate image repositories. The best associations are selected on the basis of measures of confidence. To reduce the combinatory explosion of associations, because images of the database contain very large numbers of colors and textures, we consider a visual dictionary that group together similar colors and textures.     

From content-based retrieval to invisible telepresence

Better tools for producing and managing data, combined with the human desire for information, have resulted in a data explosion. Indeed, data overload often leaves us confused, disoriented, and inefficient. The challenge is to find relevant data that lets us extract the information we want. Keyword-based systems cannot do this well, especially when working with images and video. It will be impossible to cope with the multimedia data explosion unless the data is organized for rapid information retrieval. Research in this field is in its infancy; nevertheless, commercial products are starting to appear that allow retrieval of images and video using query by pictorial example techniques. At present, these techniques work using only image primitives, but similar techniques based on domain knowledge should be available soon. Thus we will have additional techniques for providing navigational engines to ensure that the information highway is more than just a data network     

Image content-based retrieval using chromaticity moments

A number of different approaches have been recently presented for image retrieval using color features. Most of these methods use the color histogram or some variation of it. If the extracted information is to be stored for each image, such methods may require a significant amount of space for storing the histogram, depending on a given image's size and content. In the method proposed, only a small number of features, called chromaticity moments, are required to capture the spectral content (chrominance) of an image. The proposed method is based on the concept of the chromaticity diagram and extracts a set of two-dimensional moments from it to characterize the shape and distribution of chromaticities of the given image. This representation is compact (only a few chromaticity moments per image are required) and constant (independent of image size and content), while its retrieval effectiveness is comparable to using the full chromaticity histogram.     

Complementary relevance feedback-based content-based image retrieval

We propose a complementary relevance feedback-based content-based image retrieval (CBIR) system. This system exploits the synergism between short-term and long-term learning techniques to improve the retrieval performance. Specifically, we construct an adaptive semantic repository in long-term learning to store retrieval patterns of historical query sessions. We then extract high-level semantic features from the semantic repository and seamlessly integrate low-level visual features and high-level semantic features in short-term learning to effectively represent the query in a single retrieval session. The high-level semantic features are dynamically updated based on users’ query concept and therefore represent the image’s semantic concept more accurately. Our extensive experimental results demonstrate that the proposed system outperforms its seven state-of-the-art peer systems in terms of retrieval precision and storage space on a large scale imagery database.     

增量学习语义属性的图像内容检索系统增强

大规模图像内容检索是实现图像语义信息获取的重要手段, 其首要需解决图像低层特征与用户高层语义间的语义鸿沟问题。针对该问题, 引入图像语义属性, 并结合增量分类学习方法(online core vector machine, OCVM), 提出了一种增量构建大规模图像内容检索系统的新方法。该方法借助检索反馈学习机制可以提升图像语义属性的辨别准确性, 能在扩张图像库规模的同时, 提升图像内容检索的可靠性。实验结果表明了上述方法的有效性, 其检索性能可逐步地达到离线构建方法的最佳性能, 但具有更好的可扩展性和自提升能力。     

Series feature aggregation for content-based image retrieval

Feature aggregation is a critical technique in content-based image retrieval (CBIR) systems that employs multiple visual features to characterize image content. Most previous feature aggregation schemes apply parallel topology, e.g., the linear combination scheme, which suffer from two problems. First, the function of individual visual feature is limited since the ranks of the retrieved images are determined only by the combined similarity. Second, the irrelevant images seriously affect the retrieval performance of feature aggregation scheme since all images in a collection will be ranked. To address these problems, we propose a new feature aggregation scheme, series feature aggregation (SFA). SFA selects relevant images using visual features one by one in series from the images highly ranked by the previous visual feature. The irrelevant images will be effectively filtered out by individual visual features in each stage, and the remaining images are collectively described by all visual features. Experiments, conducted with IAPR TC-12 benchmark image collection (ImageCLEF2006) that contains over 20,000 photographic images and defined queries, have shown that the proposed SFA can outperform conventional parallel feature aggregation schemes.     

An adaptive technique for content-based image retrieval

We discuss an adaptive approach towards Content-Based Image Retrieval. It is based on the Ostensive Model of developing information needs—a special kind of relevance feedback model that learns from implicit user feedback and adds a temporal notion to relevance. The ostensive approach supports content-assisted browsing through visualising the interaction by adding user-selected images to a browsing path, which ends with a set of system recommendations. The suggestions are based on an adaptive query learning scheme, in which the query is learnt from previously selected images. Our approach is an adaptation of the original Ostensive Model based on textual features only, to include content-based features to characterise images. In the proposed scheme textual and colour features are combined using the Dempster-Shafer theory of evidence combination. Results from a user-centred, work-task oriented evaluation show that the ostensive interface is preferred over a traditional interface with manual query facilities. This is due to its ability to adapt to the user's need, its intuitiveness and the fluid way in which it operates. Studying and comparing the nature of the underlying information need, it emerges that our approach elicits changes in the user's need based on the interaction, and is successful in adapting the retrieval to match the changes. In addition, a preliminary study of the retrieval performance of the ostensive relevance feedback scheme shows that it can outperform a standard relevance feedback strategy in terms of image recall in category search.     

Exploiting concept clusters for content-based information retrieval

Current approaches to index weighting for information retrieval from texts are based on statistical analysis of the texts' contents. A key shortcoming of these indexing schemes, which consider only the terms in a document, is that they cannot extract semantically exact indexes that represent the semantic content of a document. To address this issue, we proposed a new indexing formalism that considers not only the terms in a document, but also the concepts. In the proposed method, concepts are extracted by exploiting clusters of terms that are semantically related, referred to as concept clusters. Through experiments on the TREC-2 collection of Wall Street Journal documents, we show that the proposed method outperforms an indexing method based on term frequency (TF), especially in regard to the highest-ranked documents. Moreover, the index term dimension was 53.3% lower for the proposed method than for the TF-based method, which is expected to significantly reduce the document search time in a real environment.     

A multi-agent platform for content-based image retrieval

Efficient and possibly intelligent image retrieval is an important task, often required in many fields of human activity. While traditional database indexing techniques exhibit a remarkable performance in textual information retrieval current research in content-based image retrieval is focused on developing novel techniques that are biologically motivated and efficient. It is well known that humans have a remarkable ability to process visual information and to handle the volume and complexity of such information quite efficiently. In this paper, we present a content-based image retrieval platform that is based on a multi-agent architecture. Each agent is responsible for assessing the similarity of the query image to each candidate image contained in a collection based on a specific primitive feature and a corresponding similarity criterion. The outputs of various agents are integrated using one of several voting schemes supported by the system. The system’s performance has been evaluated using various collections of images, as well as images obtained in specific application domains such as medical imaging. The initial evaluation has yielded very promising results.

Learning similarity matching in multimedia content-based retrieval

Many multimedia content-based retrieval systems allow query formulation with the user setting the relative importance of features (e.g., color, texture, shape, etc.) to mimic the user's perception of similarity. However, the systems do not modify their similarity matching functions, which are defined during the system development. We present a neural network-based learning algorithm for adapting the similarity matching function toward the user's query preference based on his/her relevance feedback. The relevance feedback is given as ranking errors (misranks) between the retrieved and desired lists of multimedia objects. The algorithm is demonstrated for facial image retrieval using the NIST Mugshot Identification Database with encouraging results     

Cluster-driven refinement for content-based digital image retrieval

Increasing application demands are pushing databases toward providing effective and efficient support for content-based retrieval over multimedia objects. In addition to adequate retrieval techniques, it is also important to enable some form of adaptation to users' specific needs. This paper introduces a new refinement method for retrieval based on the learning of the users' specific preferences. The proposed system indexes objects based on shape and groups them into a set of clusters, with each cluster represented by a prototype. Clustering constructs a taxonomy of objects by forming groups of closely-related objects. The proposed approach to learn the users' preferences is to refine corresponding clusters from objects provided by the users in the foreground, and to simultaneously adapt the database index in the background. Queries can be performed based solely on shape, or on a combination of shape with other features such as color. Our experimental results show that the system successfully adapts queries into databases with only a small amount of feedback from the users. The quality of the returned results is superior to that of a color-based query, and continues to improve with further use.     

A Center-Surround Histogram for content-based image retrieval

In this paper, a new type of histogram which incorporates only the visual information surrounding the edges of the image is introduced. The edge extraction operation is performed with the use of a center-surround operator of the Human Visual System. The proposed Center-Surround Histogram (CSH) has two main advantages over the classic histogram. First, it reduces the amount of visual information that needs to be processed and second, it incorporates a degree of spatial information when used in content based image retrieval applications. The method is compared with other contemporary image retrieval methods, including that of another edge color histogram, on two different databases. The comparison shows that the use of CSH exhibits better results in shorter execution times.     

Bayesian relevance feedback for content-based image retrieval

Despite the efforts to reduce the so-called semantic gap between the user's perception of image similarity and the feature-based representation of images, the interaction with the user remains fundamental to improve performances of content-based image retrieval systems. To this end, relevance feedback mechanisms are adopted to refine image-based queries by asking users to mark the set of images retrieved in a neighbourhood of the query as being relevant or not. In this paper, the Bayesian decision theory is used to estimate the boundary between relevant and non-relevant images. Then, a new query is computed whose neighbourhood is likely to fall in a region of the feature space containing relevant images. The performances of the proposed query shifting method have been compared with those of other relevance feedback mechanisms described in the literature. Reported results show the superiority of the proposed method.     

Qualitative part-based models in content-based image retrieval

A qualitative, volumetric part-based model is proposed to improve the categorical invariance and viewpoint invariance in content-based image retrieval, and a novel two-step part-categorization method is presented to build it. The method consists first in transforming parts extracted from a segmented contour primitive map and then categorizing the transformed parts using interpretation rules. The first step allows noisy extracted parts to be transformed to the domain of a simple classifier. The second step computes features of the transformed parts for categorization. Content-based image retrieval experiments using real images of complex multi-part objects confirm that a model built from the categorized parts improves both the categorical invariance and the viewpoint invariance. It does so by directly addressing the fundamental limits of low-level models.     

Relevance feature mapping for content-based multimedia information retrieval

This paper presents a novel ranking framework for content-based multimedia information retrieval (CBMIR). The framework introduces relevance features and a new ranking scheme. Each relevance feature measures the relevance of an instance with respect to a profile of the targeted multimedia database. We show that the task of CBMIR can be done more effectively using the relevance features than the original features. Furthermore, additional performance gain is achieved by incorporating our new ranking scheme which modifies instance rankings based on the weighted average of relevance feature values. Experiments on image and music databases validate the efficacy and efficiency of the proposed framework.