Visual image retrieval by elastic matching of user sketches

Effective image retrieval by content from database requires that visual image properties are used instead of textual labels to properly index and recover pictorial data. Retrieval by shape similarity, given a user-sketched template is particularly challenging, owing to the difficulty to derive a similarity measure that closely conforms to the common perception of similarity by humans. In this paper, we present a technique which is based on elastic matching of sketched templates over the shapes in the images to evaluate similarity ranks. The degree of matching achieved and the elastic deformation energy spent by the sketch to achieve such a match are used to derive a measure of similarity between the sketch and the images in the database and to rank images to be displayed. The elastic matching is integrated with arrangements to provide scale invariance and take into account spatial relationships between objects in multi-object queries. Examples from a prototype system are expounded with considerations about the effectiveness of the approach and comparative performance analysis     

A scheme of colour image retrieval from databases

This paper presents a scheme of image retrieval from a database using queries prompted by the colour and the shape of the objects present in different scenes. Of the whole scheme of image retrieval, we will focus attention on the modules that allow feature extraction of the component objects from the scenes and the matching of the objects among the different images. The defined scheme enables the indexing of images by measuring the similarity between the integral objects.     

Flexible syntactic matching of curves and its application toautomatic hierarchical classification of silhouettes

Curve matching is one instance of the fundamental correspondence problem. Our flexible algorithm is designed to match curves under substantial deformations and arbitrary large scaling and rigid transformations. A syntactic representation is constructed for both curves and an edit transformation which maps one curve to the other is found using dynamic programming. We present extensive experiments where we apply the algorithm to silhouette matching. In these experiments, we examine partial occlusion, viewpoint variation, articulation, and class matching (where silhouettes of similar objects are matched). Based on the qualitative syntactic matching, we define a dissimilarity measure and we compute it for every pair of images in a database of 121 images. We use this experiment to objectively evaluate our algorithm. First, we compare our results to those reported by others. Second, we use the dissimilarity values in order to organize the image database into shape categories. The veridical hierarchical organization stands as evidence to the quality of our matching and similarity estimation     

WALRUS: a similarity retrieval algorithm for image databases

Approaches for content-based image querying typically extract a single signature from each image based on color, texture, or shape features. The images returned as the query result are then the ones whose signatures are closest to the signature of the query image. While efficient for simple images, such methods do not work well for complex scenes since they fail to retrieve images that match the query only partially, that is, only certain regions of the image match. This inefficiency leads to the discarding of images that may be semantically very similar to the query image since they may contain the same objects. The problem becomes even more apparent when we consider scaled or translated versions of the similar objects. We propose WALRUS (wavelet-based retrieval of user-specified scenes), a novel similarity retrieval algorithm that is robust to scaling and translation of objects within an image. WALRUS employs a novel similarity model in which each image is first decomposed into its regions and the similarity measure between a pair of images is then defined to be the fraction of the area of the two images covered by matching regions from the images. In order to extract regions for an image, WALRUS considers sliding windows of varying sizes and then clusters them based on the proximity of their signatures. An efficient dynamic programming algorithm is used to compute wavelet-based signatures for the sliding windows. Experimental results on real-life data sets corroborate the effectiveness of WALRUS'S similarity model.     

A content based image retrieval system for a biological specimen collection

Digital photography and decreasing cost of storing data in digital form has led to an explosion of large digital image repositories. Since the number of images in image databases can be large (millions in some cases) it is important to develop automated tools to search them. In this paper, we present a content based image retrieval system for a database of parasite specimen images. Unlike most content based image retrieval systems, where the database consists of objects that vary widely in shape and size, the objects in our database are fairly uniform. These objects are characterized by flexible body shapes, but with fairly rigid ends. We define such shapes to be FleBoRE (Flexible Body Rigid Extremities) objects, and present a shape model for this class of objects. We have defined similarity functions to compute the degree of likeness between two FleBoRE objects and developed automated methods to extract them from specimen images. The system has been tested with a collection of parasite images from the Harold W. Manter Laboratory for Parasitology. Empirical and expert-based evaluations show that query by shape approach is effective in retrieving specimens of the same class.     

Image database design based on 9D-SPA representation for spatial relations

Spatial relationships between objects are important features for designing a content-based image retrieval system. We propose a new scheme, called 9D-SPA representation, for encoding the spatial relations in an image. With this representation, important functions of intelligent image database systems such as visualization, browsing, spatial reasoning, iconic indexing, and similarity retrieval can be easily achieved. The capability of discriminating images based on 9D-SPA representation is much more powerful than any spatial representation method based on minimum bounding rectangles or centroids of objects. The similarity measures using 9D-SPA representation provide a wide range of fuzzy matching capability in similarity retrieval to meet different user's requirements. Experimental results showed that our system is very effective in terms of recall and precision. In addition, the 9D-SPA representation can be incorporated into a two-level index structure to help reduce the search space of each query processing. The experimental results also demonstrated that, on average, only 0.1254 percent /spl sim/ 1.6829 percent of symbolic pictures (depending on various degrees of similarity) were accessed per query in an image database containing 50,000 symbolic pictures.     

Content-based image retrieval based on a fuzzy approach

A typical content-based image retrieval (CBIR) system would need to handle the vagueness in the user queries as well as the inherent uncertainty in image representation, similarity measure, and relevance feedback. We discuss how fuzzy set theory can be effectively used for this purpose and describe an image retrieval system called FIRST (fuzzy image retrieval system) which incorporates many of these ideas. FIRST can handle exemplar-based, graphical-sketch-based, as well as linguistic queries involving region labels, attributes, and spatial relations. FIRST uses fuzzy attributed relational graphs (FARGs) to represent images, where each node in the graph represents an image region and each edge represents a relation between two regions. The given query is converted to a FARG, and a low-complexity fuzzy graph matching algorithm is used to compare the query graph with the FARGs in the database. The use of an indexing scheme based on a leader clustering algorithm avoids an exhaustive search of the FARG database. We quantify the retrieval performance of the system in terms of several standard measures.     

NeTra: A toolbox for navigating large image databases

We present here an implementation of NeTra, a prototype image retrieval system that uses color, texture, shape and spatial location information in segmented image regions to search and retrieve similar regions from the database. A distinguishing aspect of this system is its incorporation of a robust automated image segmentation algorithm that allows object- or region-based search. Image segmentation significantly improves the quality of image retrieval when images contain multiple complex objects. Images are segmented into homogeneous regions at the time of ingest into the database, and image attributes that represent each of these regions are computed. In addition to image segmentation, other important components of the system include an efficient color representation, and indexing of color, texture, and shape features for fast search and retrieval. This representation allows the user to compose interesting queries such as “retrieve all images that contain regions that have the color of object A, texture of object B, shape of object C, and lie in the upper of the image”, where the individual objects could be regions belonging to different images. A Java-based web implementation of NeTra is available at http://vivaldi.ece.ucsb.edu/Netra.     

The choice of vantage objects for image retrieval

Suppose that we have a matrix of dissimilarities between n images of a database. For a new image, we would like to select the most similar image of our database. Because it may be too expensive to compute the dissimilarities for the new object to all images of our database, we want to find p?n “vantage objects” (Pattern Recognition 35 (2002) 69) from our database in order to select a matching image according to the least Euclidean distance between the vector of dissimilarities between the new image and the vantage objects and the corresponding vector for the images of the database. In this paper, we treat the choice of suitable vantage objects. We suggest a loss measure to assess the quality of a set of vantage objects: For every image, we select a matching image from the remaining images of the database by use of the vantage set, and we average the resulting dissimilarities. We compare two classes of choice strategies: The first one is based on a stepwise forward selection of vantage objects to optimize the loss measure. The second is to choose objects as representative as possible for the whole range of the database.     

An object-oriented fuzzy data model for similarity detection in image databases

We introduce a fuzzy set theoretic approach for dealing with uncertainty in images in the context of spatial and topological relations existing among the objects in the image. We propose an object-oriented graph theoretic model for representing an image and this model allows us to assess the similarity between images using the concept of (fuzzy) graph matching. Sufficient flexibility has been provided in the similarity algorithm so that different features of an image may be independently focused upon.     

Trademark image retrieval using synthetic features for describing global shape and interior structure

A trademark image retrieval (TIR) system is proposed in this work to deal with the vast number of trademark images in the trademark registration system. The proposed approach commences with the extraction of edges using the Canny edge detector, performs a shape normalisation procedure, and then extracts the global and local features. The global features capture the gross essence of the shapes while the local features describe the interior details of the trademarks. A two-component feature matching strategy is used to measure the similarity between the query and database images. The performance of the proposed algorithm is compared against four other algorithms.     

Facial expression recognition based on shape and texture

In this paper, an efficient method for human facial expression recognition is presented. We first propose a representation model for facial expressions, namely the spatially maximum occurrence model (SMOM), which is based on the statistical characteristics of training facial images and has a powerful representation capability. Then the elastic shape-texture matching (ESTM) algorithm is used to measure the similarity between images based on the shape and texture information. By combining SMOM and ESTM, the algorithm, namely SMOM-ESTM, can achieve a higher recognition performance level. The recognition rates of the SMOM-ESTM algorithm based on the AR database and the Yale database are 94.5% and 94.7%, respectively.     

Blobworld: image segmentation using expectation-maximization and its application to image querying

Retrieving images from large and varied collections using image content as a key is a challenging and important problem. We present a new image representation that provides a transformation from the raw pixel data to a small set of image regions that are coherent in color and texture. This "Blobworld" representation is created by clustering pixels in a joint color-texture-position feature space. The segmentation algorithm is fully automatic and has been run on a collection of 10,000 natural images. We describe a system that uses the Blobworld representation to retrieve images from this collection. An important aspect of the system is that the user is allowed to view the internal representation of the submitted image and the query results. Similar systems do not offer the user this view into the workings of the system; consequently, query results from these systems can be inexplicable, despite the availability of knobs for adjusting the similarity metrics. By finding image regions that roughly correspond to objects, we allow querying at the level of objects rather than global image properties. We present results indicating that querying for images using Blobworld produces higher precision than does querying using color and texture histograms of the entire image in cases where the image contains distinctive objects.     

A rotation- and flip-invariant algorithm for representing spatial continuity information of geographic images in content-based image retrieval

This research proposes a rotation- and flip-invariant algorithm for representing spatial continuity information in high-resolution geographic images for content based image retrieval (CBIR). Starting with variogram concept, the new visual property representation, in the form of a numeric index vector, consists of a set of semi-variances at selected lags and directions, based on three well-justified principles: (1) capture the basic shape of sample variogram, (2) represent the spatial continuity anisotropy, and (3) make the representation rotation- and flip-invariant. The algorithm goes through two tests. The first test confirms that it can indeed align the image representations based on spatial continuity information of objects within images by re-ordering the semi-variances accordingly. In the second test, the algorithm is applied to retrieve seven types of typical geographic entities from an Erie County orthophoto database. The retrieval results demonstrate the effectiveness of the new algorithm in CBIR, as assessed by retrieval precision.     

利用独立分量分析的运动模糊图像检索

研究了独立分量分析（ICA）算法在运动模糊图像检索中的应用。首先,对图片库中的图像进行ICA处理,构造由相互独立的基向量构成的子空间,将图片库中的图像及运动模糊图像分别向该空间投影,获得各自的特征。其次,利用特征向量间的余弦距离作为相似度度量标准,根据最近邻准则进行特征匹配与图像检索。最后,对人为加入高斯噪声、进行45°和90°旋转的运动模糊以及缺损图像进行了匹配检索实验。实验结果表明,利用ICA算法提取出的特征可以准确地检索出运动模糊图像的原图像,并且对噪声污染、旋转变换和图像缺损具有良好的鲁棒性。     

Simple low-dimensional features approximating NCC-based image matching

This paper proposes new low-dimensional image features that enable images to be very efficiently matched. Image matching is one of the key technologies for many vision-based applications, including template matching, block motion estimation, video compression, stereo vision, image/video near-duplicate detection, similarity join for image/video database, and so on. Normalized cross correlation (NCC) is one of widely used method for image matching with preferable characteristics such as robustness to intensity offsets and contrast changes, but it is computationally expensive. The proposed features, derived by the method of Lagrange multipliers, can provide upper-bounds of NCC as a simple dot product between two low-dimensional feature vectors. By using the proposed features, NCC-based image matching can be effectively accelerated. The matching performance with the proposed features is demonstrated using an image database obtained from actual broadcast videos. The new features are shown to outperform other methods: multilevel successive elimination algorithm (MSEA), discrete cosine transform (DCT) coefficients, and histograms, achieving very high precision while only slightly sacrificing recall.     

Vector space model adaptation and pseudo relevance feedback for content-based image retrieval

Image retrieval is an important problem for researchers in computer vision and content-based image retrieval (CBIR) fields. Over the last decades, many image retrieval systems were based on image representation as a set of extracted low-level features such as color, texture and shape. Then, systems calculate similarity metrics between features in order to find similar images to a query image. The disadvantage of this approach is that images visually and semantically different may be similar in the low level feature space. So, it is necessary to develop tools to optimize retrieval of information. Integration of vector space models is one solution to improve the performance of image retrieval. In this paper, we present an efficient and effective retrieval framework which includes a vectorization technique combined with a pseudo relevance model. The idea is to transform any similarity matching model (between images) to a vector space model providing a score. A study on several methodologies to obtain the vectorization is presented. Some experiments have been undertaken on Wang, Oxford5k and Inria Holidays datasets to show the performance of our proposed framework.