Efficient extraction of clustering-based feature signatures using GPU architectures

Similarity search and content-based retrieval have become widely used in multimedia database systems that often manage huge data collections. Unfortunately, many effective content-based similarity models cannot be fully utilized for larger datasets, as they are computationally demanding and require massive parallel processing for both feature extraction and query evaluation tasks. In this work, we address the performance issues of effective similarity models based on feature signatures, where we focus on fast feature extraction from image thumbnails using affordable hardware. More specifically, we propose a multi-GPU implementation that increases the extraction speed by two orders of magnitude with respect to a single-threaded CPU implementation. Since the extraction algorithm is not directly parallelizable, we propose a modification of the algorithm embracing the SIMT execution model. We have experimentally verified that our GPU extractor can be successfully used to index large image datasets comprising millions of images. In order to obtain optimal extraction parameters, we employed the GPU extractor in an extensive empirical investigation of the parameter space. The experimental results are discussed from the perspectives of both performance and similarity precision.     

Image database design based on 9D-SPA representation for spatial relations

Spatial relationships between objects are important features for designing a content-based image retrieval system. We propose a new scheme, called 9D-SPA representation, for encoding the spatial relations in an image. With this representation, important functions of intelligent image database systems such as visualization, browsing, spatial reasoning, iconic indexing, and similarity retrieval can be easily achieved. The capability of discriminating images based on 9D-SPA representation is much more powerful than any spatial representation method based on minimum bounding rectangles or centroids of objects. The similarity measures using 9D-SPA representation provide a wide range of fuzzy matching capability in similarity retrieval to meet different user's requirements. Experimental results showed that our system is very effective in terms of recall and precision. In addition, the 9D-SPA representation can be incorporated into a two-level index structure to help reduce the search space of each query processing. The experimental results also demonstrated that, on average, only 0.1254 percent /spl sim/ 1.6829 percent of symbolic pictures (depending on various degrees of similarity) were accessed per query in an image database containing 50,000 symbolic pictures.     

Complementary relevance feedback-based content-based image retrieval

We propose a complementary relevance feedback-based content-based image retrieval (CBIR) system. This system exploits the synergism between short-term and long-term learning techniques to improve the retrieval performance. Specifically, we construct an adaptive semantic repository in long-term learning to store retrieval patterns of historical query sessions. We then extract high-level semantic features from the semantic repository and seamlessly integrate low-level visual features and high-level semantic features in short-term learning to effectively represent the query in a single retrieval session. The high-level semantic features are dynamically updated based on users’ query concept and therefore represent the image’s semantic concept more accurately. Our extensive experimental results demonstrate that the proposed system outperforms its seven state-of-the-art peer systems in terms of retrieval precision and storage space on a large scale imagery database.     

An Image Database System with Support for Traditional Alphanumeric Queries and Content-Based Queries by Example

Large image databases are commonly employed in applications like criminal records, customs, plant root databases, and voters' registration databases. Efficient and convenient mechanisms for database organization and retrieval are essential. A quick and easy-to-use interface is needed which should also mesh naturally with the overall image management system. In this paper we describe the design and implementation of an integrated image database system. This system offers support for both alphanumeric query, based on alphanumeric data attached to the image file, and content-based query utilizing image examples. Content-based retrieval, specifically Query by Image Example, is made possible by the SHOSLIF approach. Alphanumeric query is implemented by a collection of parsing and query modules. All these are accessible from within a user-friendly GUI.     

Utilizing similarity relationships among existing data for high accuracy processing of content-based image retrieval

Retrieving similar images based on its visual content is an important yet difficult problem. We propose in this paper a new method to improve the accuracy of content-based image retrieval systems. Typically, given a query image, existing retrieval methods return a ranked list based on the similarity scores between the query and individual images in the database. Our method goes further by relying on an analysis of the underlying connections among individual images in the database to improve this list. Initially, we consider each image in the database as a query and use an existing baseline method to search for its likely similar images. Then, the database is modeled as a graph where images are nodes and connections among possibly similar images are edges. Next, we introduce an algorithm to split this graph into stronger subgraphs, based on our notion of graph’s strength, so that images in each subgraph are expected to be truly similar to each other. We create for each subgraph a structure called integrated image which contains the visual features of all images in the subgraph. At query time, we compute the similarity scores not only between the query and individual database images but also between the query and the integrated images. The final similarity score of a database image is computed based on both its individual score and the score of the integrated image that it belongs to. This leads effectively to a re-ranking of the retrieved images. We evaluate our method on a common image retrieval benchmark and demonstrate a significant improvement over the traditional bag-of-words retrieval model.     

Expert system for color image retrieval

Recently, as Web and various databases contain a large number of images, content-based image retrieval (CBIR) applications are greatly needed. This paper proposes a new image retrieval system using color-spatial information from those applications.First, this paper suggests two kinds of indexing keys to prune away irrelevant images to a given query image: major colors' set (MCS) signature related with color information and distribution block signature (DBS) related with spatial information. After successively applying these filters to a large database, we get only small amount of high potential candidates that are somewhat similar to a query image. Then we make use of the quad modeling (QM) method to set the initial weights of two-dimensional cell in a query image according to each major color. Finally, we retrieve more similar images from the database by comparing a query image with candidate images through a similarity measuring function associated with the weights. In that procedure, we use a new relevance feedback mechanism. This feedback enhances the retrieval effectiveness by dynamically modulating the weights of color-spatial information. Experiments show that the proposed system is not only efficient but also effective.     

An Evaluation of Color-Spatial Retrieval Techniques for Large Image Databases

In a color-spatial retrieval technique, the color information is integrated with the knowledge of the colors' spatial distribution to facilitate content-based image retrieval. Several techniques have been proposed in the literature, but these works have been developed independently without much comparison. In this paper, we present an experimental evaluation of three color-spatial retrieval techniques—the signature-based technique, the partition-based algorithm and the cluster-based method. We implemented these techniques and compare them on their retrieval effectiveness and retrieval efficiency. The experimental study is performed on an image database consisting of 12,000 images. With the proliferation of image retrieval mechanisms and the lack of extensive performance study, the experimental results can serve as guidelines in selecting a suitable technique and designing a new technique.     

Visual information retrieval system via content-based approach

In this paper a content-based image retrieval method that can search large image databases efficiently by color, texture, and shape content is proposed. Quantized RGB histograms and the dominant triple (hue, saturation, and value), which are extracted from quantized HSV joint histogram in the local image region, are used for representing global/local color information in the image. Entropy and maximum entry from co-occurrence matrices are used for texture information and edge angle histogram is used for representing shape information. Relevance feedback approach, which has coupled proposed features, is used for obtaining better retrieval accuracy. A new indexing method that supports fast retrieval in large image databases is also presented. Tree structures constructed by k-means algorithm, along with the idea of triangle inequality, eliminate candidate images for similarity calculation between query image and each database image. We find that the proposed method reduces calculation up to average 92.2 percent of the images from direct comparison.     

Content-based image retrieval by hierarchical linear subspace method

We describe a hierarchical linear subspace method to query large on-line image databases using image similarity as the basis of the queries. The method is based on the generic multimedia indexing (GEMINI) approach which is used in the IBM query through the image content search system. Our approach is demonstrated on image indexing, in which the subspaces correspond to different resolutions of the images. During content-based image retrieval, the search starts in the subspace with the lowest resolution of the images. In this subspace, the set of all possible similar images is determined. In the next subspace, additional metric information corresponding to a higher resolution is used to reduce this set. This procedure is repeated until the similar images can be determined. For evaluation we used three image databases and two different subspace sequences.     

Content-based image retrieval based on a fuzzy approach

A typical content-based image retrieval (CBIR) system would need to handle the vagueness in the user queries as well as the inherent uncertainty in image representation, similarity measure, and relevance feedback. We discuss how fuzzy set theory can be effectively used for this purpose and describe an image retrieval system called FIRST (fuzzy image retrieval system) which incorporates many of these ideas. FIRST can handle exemplar-based, graphical-sketch-based, as well as linguistic queries involving region labels, attributes, and spatial relations. FIRST uses fuzzy attributed relational graphs (FARGs) to represent images, where each node in the graph represents an image region and each edge represents a relation between two regions. The given query is converted to a FARG, and a low-complexity fuzzy graph matching algorithm is used to compare the query graph with the FARGs in the database. The use of an indexing scheme based on a leader clustering algorithm avoids an exhaustive search of the FARG database. We quantify the retrieval performance of the system in terms of several standard measures.     

Indexing chromatic and achromatic patterns for content-based colour image retrieval

In this paper, we present a method to represent achromatic and chromatic image signals independently for content-based image indexing and retrieval for image database applications. Starting from an opponent colour representation, human colour vision theories and modern digital signal processing technologies are applied to develop a compact and computationally efficient visual appearance model for coloured image patterns. We use the model to compute the statistics of achromatic and chromatic spatial patterns of colour images for indexing and content-based retrieval. Two types of colour images databases, one colour texture database and another photography colour image database are used to evaluate the performance of the developed method in content-based image indexing and retrieval. Experimental results are presented to show that the new method is superior or competitive to state-of-the-art content-based image indexing and retrieval techniques.     

Spatial inference and similarity retrieval of an intelligent image database system based on object's spanning representation

In this paper, we presented a novel image representation method to capture the information about spatial relationships between objects in a picture. Our method is more powerful than all other previous methods in terms of accuracy, flexibility, and capability of discriminating pictures. In addition, our method also provides different degrees of granularity for reasoning about directional relations in both 8- and 16-direction reference frames. In similarity retrieval, our system provides twelve types of similarity measures to support flexible matching between the query picture and the database pictures. By exercising a database containing 3600 pictures, we successfully demonstrated the effectiveness of our image retrieval system. Experiment result showed that 97.8% precision rate can be achieved while maintaining 62.5% recall rate; and 97.9% recall rate can be achieved while maintaining 51.7% precision rate. On an average, 86.1% precision rate and 81.2% recall rate can be achieved simultaneously if the threshold is set to 0.5 or 0.6. This performance is considered to be very good as an information retrieval system.     

Indexing shapes in image databases using the centroid–radii model

In content-based image retrieval systems, the content of an image such as color, shapes and textures are used to retrieve images that are similar to a query image. Most of the existing work focus on the retrieval effectiveness of using content for retrieval, i.e., study the accuracy (in terms of recall and precision) of using different representations of content. In this paper, we address the issue of retrieval efficiency, i.e., study the speed of retrieval, since a slow system is not useful for large image databases. In particular, we look at using the shape feature as the content of an image, and employ the centroid–radii model to represent the shape feature of objects in an image. This facilitates multi-resolution and similarity retrievals. Furthermore, using the model, the shape of an object can be transformed into a point in a high-dimensional data space. We can thus employ any existing high-dimensional point index as an index to speed up the retrieval of images. We propose a multi-level R-tree index, called the Nested R-trees (NR-trees) and compare its performance with that of the R-tree. Our experimental study shows that NR-trees can reduce the retrieval time significantly compared to R-tree, and facilitate similarity retrieval. We note that our NR-trees can also be used to index high-dimensional point data commonly found in many other applications.     

Efficient similarity search within user-specified projective subspaces

Many applications — such as content-based image retrieval, subspace clustering, and feature selection — may benefit from efficient subspace similarity search. Given a query object, the goal of subspace similarity search is to retrieve the most similar objects from the database, where the similarity distance is defined over an arbitrary subset of dimensions (or features) — that is, an arbitrary axis-aligned projective subspace — specified along with the query. Though much effort has been spent on similarity search in fixed subspaces, relatively little attention has been given to the problem of similarity search when the dimensions are specified at query time. In this paper, we propose new methods for the subspace similarity search problem for real-valued data. Extensive experiments are provided showing very competitive performance relative to state-of-the-art solutions.     

Relevance feature mapping for content-based multimedia information retrieval

This paper presents a novel ranking framework for content-based multimedia information retrieval (CBMIR). The framework introduces relevance features and a new ranking scheme. Each relevance feature measures the relevance of an instance with respect to a profile of the targeted multimedia database. We show that the task of CBMIR can be done more effectively using the relevance features than the original features. Furthermore, additional performance gain is achieved by incorporating our new ranking scheme which modifies instance rankings based on the weighted average of relevance feature values. Experiments on image and music databases validate the efficacy and efficiency of the proposed framework.     

一种图像检索中的灰色相关反馈算法

在交互式CBIR系统中，由于用户的查询需求常常是模糊的，因此检索结果从某种意义上说是不确定的。于是，可以将图像检索过程视为一个“灰色系统”，其中的查询向量以及图像特征的权重可视为“灰数”。基于此，该文提出了一种新的相关反馈技术，它采用“灰关联分析”理论来分析和描述“例子图像”与“相关图像”之间的关系，据此自动更新查询向量与图像特征的权重，从而更准确地描述用户的查询需求。实验结果表明，这种相关反馈算法能较好地描述用户的查询需求，显著地改善了图像检索的性能。     

Content-based image retrieval approach for biometric security using colour,texture and shape features controlled by fuzzy heuristics

In this paper, we discuss a new content-based image retrieval approach for biometric security, which is based on colour, texture and shape features and controlled by fuzzy heuristics. The proposed approach is based on the three well-known algorithms: colour histogram, texture and moment invariants. The use of these three algorithms ensures that the proposed image retrieval approach produces results which are highly relevant to the content of an image query, by taking into account the three distinct features of the image and similarity metrics based on Euclidean measure. Colour histogram is used to extract the colour features of an image. Gabor filter is used to extract the texture features and the moment invariant is used to extract the shape features of an image. The evaluation of the proposed approach is carried out using the standard precision and recall measures, and the results are compared with the well-known existing approaches. We present results which show that our proposed approach performs better than these approaches.     

Content-based indexing of multimedia databases

Content-based retrieval of multimedia database calls for content-based indexing techniques. Different from conventional databases, where data items are represented by a set of attributes of elementary data types, multimedia objects in multimedia databases are represented by a collection of features; similarity of object contents depends on context and frame of reference; and features of objects are characterized by multimodal feature measures. These lead to great challenges for content-based indexing. On the other hand, there are special requirements on content-based indexing: to support visual browsing, similarity retrieval, and fuzzy retrieval, nodes of the index should represent certain meaningful categories. That is to say that certain semantics must be added when performing indexing. ContIndex, the context-based indexing technique presented in this paper, is proposed to meet these challenges and special requirements. The indexing tree is formally defined by adapting a classification-tree concept. Horizontal links among nodes in the same level enhance the flexibility of the index. A special neural-network model, called Learning based on Experiences and Perspectives (FEP), has been developed to create node categories by fusing multimodal feature measures. It brings into the index the capability of self-organizing nodes with respect to certain context and frames of reference. An icon image is generated for each intermediate node to facilitate visual browsing. Algorithms have been developed to support multimedia object archival and retrieval using Contlndex     

Virtual images for similarity retrieval in image databases

We introduce the virtual image, an iconic index suited for pictorial information access in a pictorial database, and a similarity retrieval approach based on virtual images to perform content-based retrieval. A virtual image represents the spatial information contained in a real image in explicit form by means of a set of spatial relations. This is useful to efficiently compute the similarity between a query and an image in the database. We also show that virtual images support real-world applications that require translation, reflection, and/or rotation invariance of image representation