A Scheme for Intelligent Image Retrieval in Multimedia Databases

Storage and retrieval of visual data play an important role in multimedia systems. We have developed a content-based scheme for retrieving images from multimedia databases intelligently. The retrieval takes two stages. The first stage retrieves an image based on partial information. In the second stage, the system accumulates knowledge from the results of the first-stage retrieval. It analyzes the subspace of features from the resulting images and tries to understand the query request. It also makes full use of the entire index space, although queries can be made on partial information. The technology developed will find many applications in multimedia areas. It will also provide a tool for studying how humans rank the similarity of images and what information people use in visual perception, etc., and will help in the development of methods based on these human approaches.     

A new way for multidimensional medical data management: volume of interest (VOI)-based retrieval of medical images with visual and functional features.

The advances in digital medical imaging and storage in integrated databases are resulting in growing demands for efficient image retrieval and management. Content-based image retrieval (CBIR) refers to the retrieval of images from a database, using the visual features derived from the information in the image, and has become an attractive approach to managing large medical image archives. In conventional CBIR systems for medical images, images are often segmented into regions which are used to derive two-dimensional visual features for region-based queries. Although such approach has the advantage of including only relevant regions in the formulation of a query, medical images that are inherently multidimensional can potentially benefit from the multidimensional feature extraction which could open up new opportunities in visual feature extraction and retrieval. In this study, we present a volume of interest (VOI) based content-based retrieval of four-dimensional (three spatial and one temporal) dynamic PET images. By segmenting the images into VOIs consisting of functionally similar voxels (e.g., a tumor structure), multidimensional visual and functional features were extracted and used as region-based query features. A prototype VOI-based functional image retrieval system (VOI-FIRS) has been designed to demonstrate the proposed multidimensional feature extraction and retrieval. Experimental results show that the proposed system allows for the retrieval of related images that constitute similar visual and functional VOI features, and can find potential applications in medical data management, such as to aid in education, diagnosis, and statistical analysis.     

Combining positive and negative examples in relevance feedback for content-based image retrieval

In this paper, we address some issues related to the combination of positive and negative examples to improve the efficiency of image retrieval. We start by analyzing the relevance of the negative example and how it can be interpreted and utilized to mitigate certain problems in image retrieval, such as noise, miss, the page zero problem and feature selection. Then we propose a new relevance feedback approach that uses the positive example (PE) to perform generalization and the negative example (NE) to perform specialization. In this approach, a query containing both PE and NE is processed in two steps. The first step considers the PE alone, in order to reduce the set of images participating in retrieval to a more homogeneous subset. Then, the second step considers both PE and NE and acts on the images retained in the first step. Mathematically, relevance feedback is formulated as an optimization of the intra and inter variances of the PE and NE. The proposed relevance feedback algorithm was implemented in our image retrieval system, which we tested on a collection of more than 10,000 images. The experimental results show how the NE as considered in our model can contribute in improving the relevance of the images retrieved.     

基于“点到面”策略的图像检索

图像检索是计算机视觉领域的一个重要分支。其主要目的是从图像数据库中找出与查询图像相似的语义图像。传统的图像检索方法是在查询图像和数据库图像之间进行“点到点”检索。但是,单个查询图像包含的类别提示较少,即类别信息较弱,使得检索结果并不理想。为了解决这个问题,本文提出了一种基于“点到面”的类别检索策略来扩展一个图像（点）到一个图像类别（面）,这意味着从单个查询图像到整个图像类别的语义扩展。该方法挖掘了查询图像的类别信息。在两个常用的数据集上对所提出方法的性能进行了评估。实验表明,该方法可以显著提高图像检索的性能。      

Query Expansion by Text and Image Features in Image Retrieval

We present a two-pass image retrieval system in which retrieval techniques for text and image documents are combined in a novel approach. In the first pass, the text-based initial query is matched against the text captions of the images in the database to obtain the initial retrieved set. In the second pass, text and image features obtained from this initial retrieved set are used to expand the initial query. Additional images from the database are then retrieved based on the expanded query. The image features that we have used are color histograms, DC coefficients from the discrete cosine transform, and two texture features: multiresolution simultaneous autoregressive model and local binary pattern. These are low-level statistical image features that can be easily computed. Extensive experiments have been performed on 1019 color pictures of mixed variety with captions, relevance judgments and queries supplied by a national archives agency. Objective precision-recall results have been obtained with various combinations of text and image features. The results show that the image features do not perform well when used on their own. However, when image features are used in query expansion, they increase the average precision more significantly than text annotations. Moreover, these findings are valid at all precision levels and are not sensitive to the image feature acquisition parameters.     

Content-based microscopic image retrieval system for multi-image queries

In this paper, we describe the design and development of a multitiered content-based image retrieval (CBIR) system for microscopic images utilizing a reference database that contains images of more than one disease. The proposed CBIR system uses a multitiered approach to classify and retrieve microscopic images involving their specific subtypes, which are mostly difficult to discriminate and classify. This system enables both multi-image query and slide-level image retrieval in order to protect the semantic consistency among the retrieved images. New weighting terms, inspired from information retrieval theory, are defined for multiple-image query and retrieval. The performance of the system was tested on a dataset including 1666 imaged high power fields extracted from 57 follicular lymphoma (FL) tissue slides with three subtypes and 44 neuroblastoma (NB) tissue slides with four subtypes. Each slide is semantically annotated according to their subtypes by expert pathologists. By using leave-one-slide out testing scheme, the multi-image query algorithm with the proposed weighting strategy achieves about 93% and 86% of average classification accuracy at the first rank retrieval, outperforming the image-level retrieval accuracy by about 38 and 26 percentage points, for FL and NB diseases, respectively.     

Region-based image retrieval using color-size features of watershed regions

This paper presents a region-based image retrieval system that provides a user interface for helping to specify the watershed regions of interest within a query image. We first propose a new type of visual features, called color-size feature, which includes color-size histogram and moments, to integrate color and region-size information of watershed regions. Next, we design a scheme of region filtering that is based on color-size histogram to fast screen out some of most irrelevant regions and images for the preprocessing of the image retrieval. Our region-based image retrieval system applies the Earth Mover’s Distance in the design of the similarity measure for image ranking and matching. Finally, we present some experiments for the color-size feature, region filtering, and retrieval results that demonstrate the efficiency of our proposed system.     

A similarity learning approach to content-based image retrieval: application to digital mammography

In this paper, we describe an approach to content-based retrieval of medical images from a database, and provide a preliminary demonstration of our approach as applied to retrieval of digital mammograms. Content-based image retrieval (CBIR) refers to the retrieval of images from a database using information derived from the images themselves, rather than solely from accompanying text indices. In the medical-imaging context, the ultimate aim of CBIR is to provide radiologists with a diagnostic aid in the form of a display of relevant past cases, along with proven pathology and other suitable information. CBIR may also be useful as a training tool for medical students and residents. The goal of information retrieval is to recall from a database information that is relevant to the user's query. The most challenging aspect of CBIR is the definition of relevance (similarity), which is used to guide the retrieval machine. In this paper, we pursue a new approach, in which similarity is learned from training examples provided by human observers. Specifically, we explore the use of neural networks and support vector machines to predict the user's notion of similarity. Within this framework we propose using a hierarchal learning approach, which consists of a cascade of a binary classifier and a regression module to optimize retrieval effectiveness and efficiency. We also explore how to incorporate online human interaction to achieve relevance feedback in this learning framework. Our experiments are based on a database consisting of 76 mammograms, all of which contain clustered microcalcifications (MCs). Our goal is to retrieve mammogram images containing similar MC clusters to that in a query. The performance of the retrieval system is evaluated using precision-recall curves computed using a cross-validation procedure. Our experimental results demonstrate that: 1) the learning framework can accurately predict the perceptual similarity reported by human observers, thereby serving as a basis for CBIR; 2) the learning-based framework can significantly outperform a simple distance-based similarity metric; 3) the use of the hierarchical two-stage network can improve retrieval performance; and 4) relevance feedback can be effectively incorporated into this learning framework to achieve improvement in retrieval precision based on online interaction with users; and 5) the retrieved images by the network can have predicting value for the disease condition of the query.     

Histological image retrieval based on semantic content analysis

The demand for automatic recognition and retrieval of medical images for screening, reference, and management is increasing. We present an intelligent content-based image retrieval system called I-Browse, which integrates both iconic and semantic content for histological image analysis. The I-Browse system combines low-level image processing technology with high-level semantic analysis of medical image content through different processing modules in the proposed system architecture. Similarity measures are proposed and their performance is evaluated. Furthermore, as a byproduct of semantic analysis, I-Browse allows textual annotations to be generated for unknown images. As an image browser, apart from retrieving images by image example, it also supports query by natural language.     

Information mining in remote sensing image archives: system evaluation

We present an algorithmic protocol for the evaluation of a content-based remote sensing image information mining system. In order to provide users fast access to the content of large image databases, the system is composed of two main modules. The first includes computationally intensive algorithms for off-line data ingestion in the archive, image feature extraction, and indexing. The second module consists of a graphical man-machine interface that manages the information fusion for interactive interpretation and the image information mining functions. According to the system architecture, the proposed evaluation methodology aims to determine the objective technical quality of the system and includes subjective human factors as well. Since the query performance of a content-based image retrieval system mainly depends on the datasets stored in the archive, we first analyze the complexity of image data. Then, we determine the accuracy of the interactive training that can be considered as a supervised Bayesian classification of the entire archive. Based on the stochastic nature of user-defined cover types, the system retrieves images using probabilistic measurements. The information quality of the queried results is measured by target and misclassified images, precision and recall, and the probability to forget and to overretrieve images. Since the queried images are the result of a number of interactions between user and system, we analyze the man-machine communication dialogue and the system operation, too. Finally, we compare the objective component of the evaluation protocol with the users' degree of satisfaction to point out the significance of the computed measurements.     

Indexing and retrieval of images by spatial constraints

Many multimedia applications require retrieval of spatially similar images against a given query image. Existing work on image retrieval and indexing either requires extensive low-level computations or elaborate human interaction. In this paper, we introduce a new symbolic image representation technique to eliminate repetitive tasks of image understanding and object processing. Our symbolic image representation scheme is based on the concept of hierarchical decomposition of image space into spatial arrangements of features while preserving the spatial relationships among the image objects. Quadtrees are used to manage the decomposition hierarchy and play an important role in defining the similarity measure. This scheme is incremental in nature, can be adopted to accommodate varying levels of details in a wide range of application domains, and provides geometric variance independence. While ensuring that there are no false negatives, our approach also discriminates against non-matching entities by eliminating them as soon as possible, during the coarser matching phases. A hierarchical indexing scheme based on the concept of image signatures and efficient quadtree matching has been devised. Each level of the hierarchy tends to reduce the search space, allowing more involved comparisons only for potentially matching candidate database images. For a given query image, a facility is provided to rank-order the retrieved spatially similar images from the image database for subsequent browsing and selection by the user.     

Development of Content-Based Trademark Retrieval System on the World Wide Web

In this paper, we describe a new trademark retrieval system based upon the content or the shape of trademark. The system has an on-line graphical user interface for the World Wide Web (WWW) that allows user to provide a query in forms of a sketch or a visual image to search for similar trademarks from database. User interfaces for the WWW were implemented by utilizing HTML and Java applets. The query can occur in arbitrary size and orientation. A shape representation scheme invariant to scale and rotation was developed to measure the similarity between two trademarks using the magnitude of Zernike moments as a feature set. Performance evaluation has been carried out with a database of 3,000 trademarks. It takes only about 0.6 second for the retrieval on a 200 MHz Pentium PC. The average recall of the original one among top 30 candidates queried by noisy or deformed images was 100%.     

CLUE: cluster-based retrieval of images by unsupervised learning.

In a typical content-based image retrieval (CBIR) system, target images (images in the database) are sorted by feature similarities with respect to the query. Similarities among target images are usually ignored. This paper introduces a new technique, cluster-based retrieval of images by unsupervised learning (CLUE), for improving user interaction with image retrieval systems by fully exploiting the similarity information. CLUE retrieves image clusters by applying a graph-theoretic clustering algorithm to a collection of images in the vicinity of the query. Clustering in CLUE is dynamic. In particular, clusters formed depend on which images are retrieved in response to the query. CLUE can be combined with any real-valued symmetric similarity measure (metric or nonmetric). Thus, it may be embedded in many current CBIR systems, including relevance feedback systems. The performance of an experimental image retrieval system using CLUE is evaluated on a database of around 60,000 images from COREL. Empirical results demonstrate improved performance compared with a CBIR system using the same image similarity measure. In addition, results on images returned by Google's Image Search reveal the potential of applying CLUE to real-world image data and integrating CLUE as a part of the interface for keyword-based image retrieval systems.     

Efficient privacy-preserving image retrieval scheme over outsourced data with multi-user

The traditional privacy-preserving image retrieval schemes not only bring large computational and communication overhead,but also cannot protect the image and query privacy in multi-user scenarios.To solve above problems,an efficient privacy-preserving content-based image retrieval scheme was proposed in multi-user scenarios.The scheme used Euclidean distance comparison technique to rank the pictures according to similarity of picture feature vectors and return top-k returned.Meanwhile,the efficient key conversion protocol designed in proposed image retrieval scheme allowed each search user to generate queries based on his own private key so that he can retrieval encrypted images generated by different data owners.Strict security analysis shows that the user privacy and cloud data security can be well protected during the image retrieval process,and the performance analysis using real-world dataset shows that the proposed image retrieval scheme is efficient and feasible in practical applications.     

Multiple classifiers for color flag and trademark image retrieval

A novel region-based multiple classifier color image retrieval system is presented. In our approach, a region-growing technique is first employed to cluster connected color pixels with the same color in an image to form color regions which are the primitive elements utilized in our proposed approach. Then, three complementary region-based classifiers that we developed are selected in the classifier selection stage, which include color classifier, shape classifier, and relational classifier. In each classifier, a virtue probability representing the probability that an image is similar to the query image is defined. Thereafter a set of virtue probabilities is calculated in each classifier. Next, the measurement dependent methods are applied to combine the virtue probabilities of classifiers in the decision combination stage. The dynamic selection scheme designed in the decision combination stage can further improve the system performance dramatically. Experimental results reveal the feasibility and validity of our proposed approach in solving the color image retrieval problem     

Dictionary-based color image retrieval using multiset theory

Dictionaries have recently attracted a great deal of interest as a new powerful representation scheme that can describe the visual content of an image. Most existing approaches nevertheless, neglect dictionary statistics. In this work, we explore the linguistic and statistical properties of dictionaries in an image retrieval task, representing the dictionary as a multiset. This is extracted by means of the LZW data compressor which encodes the visual patterns of an image. For this reason the image is first quantized and then transformed into a 1D string of characters. Based on the multiset notion we also introduce the Normalized Multiset Distance (NMD), as a new dictionary-based dissimilarity measure which enables the user to retrieve images with similar content to a given query. Experimental results demonstrate a significant improvement in retrieval performance compared to related dictionary-based techniques or to several other image indexing methods that utilize classical low-level image features.