A formal framework for image indexing with triples: Toward a concept-based image retrieval

This paper provides a formal specification for concept-based image retrieval using triples. To effectively manage a vast amount of images, we may need an image retrieval system capable of indexing and searching images based on the characteristics of their content. However, such a content-based image retrieval technique alone may not satisfy user queries if retrieved images turn out to be relevant only when they are conceptually related with the queries. In this paper, we develop an image retrieval mechanism to extract semantics of images based on triples. The semantics can be captured by deriving concepts from its constituent objects and spatial relationships between them. The concepts are basically composite objects formed from the aggregation of the constituents. In our mechanism, all the spatial relationships between objects including the concepts are uniformly represented by triples, which are used for indexing images as well as capturing their semantics. We also develop a query evaluation for supporting the concept-based image retrieval. ©1999 John Wiley & Sons, Inc.     

An efficient similarity metric for omnidirectional vision sensors

This paper presents an efficient metric for the computation of the similarity among omnidirectional images (image matching). The representation of image appearance is based on feature vectors that include both the chromatic attributes of color sets and their mutual spatial relationships. The proposed metric fits well to robotic navigation using omnidirectional vision sensors, because it has very important properties: it is reflexive, compositional and invariant with respect to image scaling and rotation. The robustness of the metric was repeatedly tested using omnidirectional images for a robot localization task in a real indoor environment.     

Efficient Content-Based Image Retrieval through Metric Histograms

This paper presents a new and efficient method for content-based image retrieval employing the color distribution of images. This new method, called metric histogram, takes advantage of the correlation among adjacent bins of histograms, reducing the dimensionality of the feature vectors extracted from images, leading to faster and more flexible indexing and retrieval processes. The proposed technique works on each image independently from the others in the dataset, therefore there is no pre-defined number of color regions in the resulting histogram. Thus, it is not possible to use traditional comparison algorithms such as Euclidean or Manhattan distances. To allow the comparison of images through the new feature vectors given by metric histograms, a new metric distance function MHD( ) is also proposed. This paper shows the improvements in timing and retrieval discrimination obtained using metric histograms over traditional ones, even when using images with different spatial resolution or thumbnails. The experimental evaluation of the new method, for answering similarity queries over two representative image databases, shows that the metric histograms surpass the retrieval ability of traditional histograms because they are invariant on geometrical and brightness image transformations, and answer the queries up to 10 times faster than the traditional ones.     

Joint prototype and metric learning for image set classification: Application to video face identification

In this paper, we address the problem of image set classification, where each set contains a different number of images acquired from the same subject. In most of the existing literature, each image set is modeled using all its available samples. As a result, the corresponding time and storage costs are high. To address this problem, we propose a joint prototype and metric learning approach. The prototypes are learned to represent each gallery image set using fewer samples without affecting the recognition performance. A Mahalanobis metric is learned simultaneously to measure the similarity between sets more accurately. In particular, each gallery set is represented as a regularized affine hull spanned by the learned prototypes. The set-to-set distance is optimized via updating the prototypes and the Mahalanobis metric in an alternating manner. To highlight the importance of representing image sets using fewer samples, we analyzed the corresponding test time complexity with respect to the number of images used per set. Experimental results using YouTube Celebrity, YouTube Faces, and ETH-80 datasets illustrate the efficiency on the task of video face recognition, and object categorization.     

Efficient matching and indexing of graph models in content-basedretrieval

In retrieval from image databases, evaluation of similarity, based both on the appearance of spatial entities and on their mutual relationships, depends on content representation based on attributed relational graphs. This kind of modeling entails complex matching and indexing, which presently prevents its usage within comprehensive applications. In this paper, we provide a graph-theoretical formulation for the problem of retrieval based on the joint similarity of individual entities and of their mutual relationships and we expound its implications on indexing and matching. In particular, we propose the usage of metric indexing to organize large archives of graph models, and we propose an original look-ahead method which represents an efficient solution for the (sub)graph error correcting isomorphism problem needed to compute object distances. Analytic comparison and experimental results show that the proposed lookahead improves the state-of-the-art in state-space search methods and that the combined use of the proposed matching and indexing scheme permits for the management of the complexity of a typical application of retrieval by spatial arrangement     

Objective image quality measurement by local spatial‐frequency wavelet analysis

An essential determinant of the value of surrogate digital images is their quality. Image quality measurement has become crucial for most image processing applications. Over the past years, there have been many attempts to develop models or metrics for image quality that incorporate elements of human visual sensitivity. However, there is no current standard and objective definition of spectral image quality. This paper proposes a reliable automatic method for objective image quality measurement by local spatial‐frequency wavelet analysis. The analysis is performed locally by dividing an image into 64×64 pixel blocks, and transforming the data into wavelet domain and sub‐band image domain. A fast lifting wavelet algorithm is developed for computationally efficient spatial‐frequency deconstruction of images to extract the features of edges in sub‐band datasets. Wavelet analysis throughout the spatial‐frequency range with respect to noise, sharpness, brightness, contrast, and modulation transfer function (MTF) reveals more detailed information to relate the quality of an image to the interpretation by using an artificial neural network to optimize weightings and results in a promising quantitative measure of image quality. Experimental results from using this method for image quality measurement exhibit good correlation to subjective visual quality assessments. The proposed approach with fast computation provides a potential metric for reliable image quality quantification, allowing image quality to be automatically evaluated following image capture/compression, but prior to permanent storage in a database.     

Content-based image retrieval by hierarchical linear subspace method

We describe a hierarchical linear subspace method to query large on-line image databases using image similarity as the basis of the queries. The method is based on the generic multimedia indexing (GEMINI) approach which is used in the IBM query through the image content search system. Our approach is demonstrated on image indexing, in which the subspaces correspond to different resolutions of the images. During content-based image retrieval, the search starts in the subspace with the lowest resolution of the images. In this subspace, the set of all possible similar images is determined. In the next subspace, additional metric information corresponding to a higher resolution is used to reduce this set. This procedure is repeated until the similar images can be determined. For evaluation we used three image databases and two different subspace sequences.     

Symbolic picture retrieval by relative–metric spatial relations

Symbolic pictures can be used for iconic indexing, spatial reasoning, and similarity retrieval in the design of intelligent image database systems. [S. K. Chang, C. W. Yan, Donald C. Dimitroff, and Timothy Arndt, IEEE Trans. Software Engineering 1988, 14, 681–688; S.‐K. Chang, Principles of Pictorial Information Systems Design, Prentice‐Hall, New York, 1989.] However, previous approaches to designing such systems usually ignore relative‐metric information on symbolic pictures and cause several deficiencies in indexing, spatial reasoning, and retrieval. In our approach, we extract relative‐metric information from symbolic pictures and use such information to help establish indexes based on an improvement from a minimal perfect hashing scheme. As a result, more accurate picture retrieval can be achieved through our indexing mechanism. Capabilities in spatial reasoning and query representation/processing are also improved. By utilizing relative‐metric spatial relations, an image database system becomes more flexible and intelligent. ©2000 John Wiley & Sons, Inc.     

Integrated spatial and feature image query

We present a new system for querying for images by regions and their spatial and feature attributes. The system enables the user to find the images that contain arrangements of regions similar to those diagrammed in a query image. By indexing the attributes of regions, such as sizes, locations and visual features, a wide variety of complex joint spatial and feature queries are efficiently computed. In order to demonstrate the utility of the system, we develop a process for the extracting color regions from photographic images. We demonstrate that integrated spatial and feature querying using color regions improves image search capabilities over non-spatial content-based image retrieval methods.     

An efficient iconic indexing strategy for image rotation and reflection in image databases

Spatial relationships are important issues for similarity-based retrieval in many image database applications. With the popularity of digital cameras and the related image processing software, a sequence of images are often rotated or flipped. That is, those images are transformed in the rotation orientation or the reflection direction. However, many iconic indexing strategies based on symbolic projection are sensitive to rotation or reflection. Therefore, these strategies may miss the qualified images, when the query is issued in the orientation different from the orientation of the database images. To solve this problem, some researchers proposed a function to map the spatial relationship to its transformed one. However, this mapping consists of several conditional statements, which is time-consuming. Thus, in this paper, we propose an efficient iconic indexing strategy, in which we carefully assign a unique bit pattern to each spatial relationship and record the spatial information based on the bit patterns in a matrix. Without generating the rotated or flipped image, we can directly derive the index of the rotated or flipped image from the index of the original one by bit operations and matrix manipulation. In our performance study, we analyze the time complexity of our proposed strategy and show the efficiency of our proposed strategy according to the simulation results. Moreover, we implement a prototype to validate our proposed strategy.     

基于空间特征的图像检索

提出一种新的基于空间特征的图像特征描述子SCH,利用基于颜色向量角和欧几里得距离的MCVAE算法共同检测原始彩色图像边缘,同时利用一种新的“最大最小分量颜色不变量模型”对原始图像量化,对边缘像素建立边缘相关矩阵;对非边缘像素使用颜色直方图描述局部颜色分布信息;然后,利用新的sin相似性度量法则衡量图像特征间的相似度。实验采用VC++6.0开发了基于内容的图像检索原型系统“SttImageRetrieval”,基于Oracle 9i数据库建立了一个综合型图像数据库“IMAGEDB”。实验分析结果证明,利用SCH描述子的检索准确度明显高于仅基于颜色统计特征的检索结果。     

Learning multi-task local metrics for image annotation

The goal of image annotation is to automatically assign a set of textual labels to an image to describe the visual contents thereof. Recently, with the rapid increase in the number of web images, nearest neighbor (NN) based methods have become more attractive and have shown exciting results for image annotation. One of the key challenges of these methods is to define an appropriate similarity measure between images for neighbor selection. Several distance metric learning (DML) algorithms derived from traditional image classification problems have been applied to annotation tasks. However, a fundamental limitation of applying DML to image annotation is that it learns a single global distance metric over the entire image collection and measures the distance between image pairs in the image-level. For multi-label annotation problems, it may be more reasonable to measure similarity of image pairs in the label-level. In this paper, we develop a novel label prediction scheme utilizing multiple label-specific local metrics for label-level similarity measure, and propose two different local metric learning methods in a multi-task learning (MTL) framework. Extensive experimental results on two challenging annotation datasets demonstrate that 1) utilizing multiple local distance metrics to learn label-level distances is superior to using a single global metric in label prediction, and 2) the proposed methods using the MTL framework to learn multiple local metrics simultaneously can model the commonalities of labels, thereby facilitating label prediction results to achieve state-of-the-art annotation performance.     

Indexing chromatic and achromatic patterns for content-based colour image retrieval

In this paper, we present a method to represent achromatic and chromatic image signals independently for content-based image indexing and retrieval for image database applications. Starting from an opponent colour representation, human colour vision theories and modern digital signal processing technologies are applied to develop a compact and computationally efficient visual appearance model for coloured image patterns. We use the model to compute the statistics of achromatic and chromatic spatial patterns of colour images for indexing and content-based retrieval. Two types of colour images databases, one colour texture database and another photography colour image database are used to evaluate the performance of the developed method in content-based image indexing and retrieval. Experimental results are presented to show that the new method is superior or competitive to state-of-the-art content-based image indexing and retrieval techniques.     

Central catadioptric image processing with geodesic metric

Because of the distortions produced by the insertion of a mirror, catadioptric images cannot be processed similarly to classical perspective images. Now, although the equivalence between such images and spherical images is well known, the use of spherical harmonic analysis often leads to image processing methods which are more difficult to implement. In this paper, we propose to define catadioptric image processing from the geodesic metric on the unitary sphere. We show that this definition allows to adapt very simply classical image processing methods. We focus more particularly on image gradient estimation, interest point detection, and matching. More generally, the proposed approach extends traditional image processing techniques based on Euclidean metric to central catadioptric images. We show in this paper the efficiency of the approach through different experimental results and quantitative evaluations.     

Image quality assessment by discrete orthogonal moments

This paper proposes a novel full-reference quality assessment (QA) metric that automatically assesses the quality of an image in the discrete orthogonal moments domain. This metric is constructed by representing the spatial information of an image using low order moments. The computation, up to fourth order moments, is performed on each individual (8×8) non-overlapping block for both the test and reference images. Then, the computed moments of both the test and reference images are combined in order to determine the moment correlation index of each block in each order. The number of moment correlation indices used in this study is nine. Next, the mean of each moment correlation index is computed and thereafter the single quality interpretation of the test image with respect to its reference is determined by taking the mean value of the computed means of all the moment correlation indices. The proposed objective metrics based on two discrete orthogonal moments, Tchebichef and Krawtchouk moments, are developed and their performances are evaluated by comparing them with subjective ratings on several publicly available databases. The proposed discrete orthogonal moments based metric performs competitively well with the state-of-the-art models in terms of quality prediction while outperforms them in terms of computational speed.     

Expert system for color image retrieval

Recently, as Web and various databases contain a large number of images, content-based image retrieval (CBIR) applications are greatly needed. This paper proposes a new image retrieval system using color-spatial information from those applications.First, this paper suggests two kinds of indexing keys to prune away irrelevant images to a given query image: major colors' set (MCS) signature related with color information and distribution block signature (DBS) related with spatial information. After successively applying these filters to a large database, we get only small amount of high potential candidates that are somewhat similar to a query image. Then we make use of the quad modeling (QM) method to set the initial weights of two-dimensional cell in a query image according to each major color. Finally, we retrieve more similar images from the database by comparing a query image with candidate images through a similarity measuring function associated with the weights. In that procedure, we use a new relevance feedback mechanism. This feedback enhances the retrieval effectiveness by dynamically modulating the weights of color-spatial information. Experiments show that the proposed system is not only efficient but also effective.