A unified framework for image retrieval using keyword and visual features.

In this paper, a unified image retrieval framework based on both keyword annotations and visual features is proposed. In this framework, a set of statistical models are built based on visual features of a small set of manually labeled images to represent semantic concepts and used to propagate keywords to other unlabeled images. These models are updated periodically when more images implicitly labeled by users become available through relevance feedback. In this sense, the keyword models serve the function of accumulation and memorization of knowledge learned from user-provided relevance feedback. Furthermore, two sets of effective and efficient similarity measures and relevance feedback schemes are proposed for query by keyword scenario and query by image example scenario, respectively. Keyword models are combined with visual features in these schemes. In particular, a new, entropy-based active learning strategy is introduced to improve the efficiency of relevance feedback for query by keyword. Furthermore, a new algorithm is proposed to estimate the keyword features of the search concept for query by image example. It is shown to be more appropriate than two existing relevance feedback algorithms. Experimental results demonstrate the effectiveness of the proposed framework.     

A memory learning framework for effective image retrieval.

Most current content-based image retrieval systems are still incapable of providing users with their desired results. The major difficulty lies in the gap between low-level image features and high-level image semantics. To address the problem, this study reports a framework for effective image retrieval by employing a novel idea of memory learning. It forms a knowledge memory model to store the semantic information by simply accumulating user-provided interactions. A learning strategy is then applied to predict the semantic relationships among images according to the memorized knowledge. Image queries are finally performed based on a seamless combination of low-level features and learned semantics. One important advantage of our framework is its ability to efficiently annotate images and also propagate the keyword annotation from the labeled images to unlabeled images. The presented algorithm has been integrated into a practical image retrieval system. Experiments on a collection of 10,000 general-purpose images demonstrate the effectiveness of the proposed framework.     

A framework for medical image retrieval using machine learning and statistical similarity matching techniques with relevance feedback.

A content-based image retrieval (CBIR) framework for diverse collection of medical images of different imaging modalities, anatomic regions with different orientations and biological systems is proposed. Organization of images in such a database (DB) is well defined with predefined semantic categories; hence, it can be useful for category-specific searching. The proposed framework consists of machine learning methods for image prefiltering, similarity matching using statistical distance measures, and a relevance feedback (RF) scheme. To narrow down the semantic gap and increase the retrieval efficiency, we investigate both supervised and unsupervised learning techniques to associate low-level global image features (e.g., color, texture, and edge) in the projected PCA-based eigenspace with their high-level semantic and visual categories. Specially, we explore the use of a probabilistic multiclass support vector machine (SVM) and fuzzy c-mean (FCM) clustering for categorization and prefiltering of images to reduce the search space. A category-specific statistical similarity matching is proposed in a finer level on the prefiltered images. To incorporate a better perception subjectivity, an RF mechanism is also added to update the query parameters dynamically and adjust the proposed matching functions. Experiments are based on a ground-truth DB consisting of 5000 diverse medical images of 20 predefined categories. Analysis of results based on cross-validation (CV) accuracy and precision-recall for image categorization and retrieval is reported. It demonstrates the improvement, effectiveness, and efficiency achieved by the proposed framework.     

An efficient and effective region-based image retrieval framework

An image retrieval framework that integrates efficient region-based representation in terms of storage and complexity and effective on-line learning capability is proposed. The framework consists of methods for region-based image representation and comparison, indexing using modified inverted files, relevance feedback, and learning region weighting. By exploiting a vector quantization method, both compact and sparse (vector) region-based image representations are achieved. Using the compact representation, an indexing scheme similar to the inverted file technology and an image similarity measure based on Earth Mover's Distance are presented. Moreover, the vector representation facilitates a weighted query point movement algorithm and the compact representation enables a classification-based algorithm for relevance feedback. Based on users' feedback information, a region weighting strategy is also introduced to optimally weight the regions and enable the system to self-improve. Experimental results on a database of 10,000 general-purposed images demonstrate the efficiency and effectiveness of the proposed framework.     

Contextual aerial image categorization using codebook

Effective categorization of the millions of aerial images from unmanned planes is a useful technique with several important applications. Previous methods on this task usually encountered such problems: (1) it is hard to represent the aerial images’ topologies efficiently, which are the key feature to distinguish the arial images rather than conventional appearance, and (2) the computational load is usually too high to build a realtime image categorization system. Addressing these problems, this paper proposes an efficient and effective aerial image categorization method based on a contextual topological codebook. The codebook of aerial images is learned with a multitask learning framework. The topology of each aerial image is represented with the region adjacency graph (RAG). Furthermore, a codebook containing topologies is learned by jointly modeling the contextual information, based on the extracted discriminative graphlets. These graphlets are integrated into a Bag-of-Words (BoW) representation for predicting aerial image categories. Contextual relation among local patches are taken into account in categorization to yield high categorization performance. Experimental results show that our approach is both effective and efficient.     

基于局部二值模式的医学图像检索

提出了一种基于局部二值模式(LBP)和纹理模式统计进行医学图像检索的方法,计算了LBP和局部方差的联合直方图,改进了Log-likelihood统计距离度量算法.通过仿真表明:改进的Log-likelihood统计算法比Log-likelihood统计算法检索准确率高:与基于Gabor纹理特征图像检索相比较,该局部二值纹理模式检索算法检索准确率能提高8%以上.     

Improving retrieval framework using information gain models

Content-based image retrieval systems are meant to retrieve the most similar images of a collection to a query image. One of the most well-known models widely applied for this task is the bag of visual words (BoVW) model. In this paper, we introduce a study of different information gain models used for the construction of a visual vocabulary. In the proposed framework, information gain models are used as a discriminative information to index image features and select the ones that have the highest information gain values. We introduce some extensions to further improve the performance of the proposed framework: mixing different vocabularies and extending the BoVW to bag of visual phrases. Exhaustive experiments show the interest of information gain models on our retrieval framework.     

Content-based image retrieval using H.264 intra coding features

Efficient multimedia retrieval has become a vital issue because more audio and video data are now available. This paper focuses on content-based image retrieval (CBIR) in the compression domain (CPD). The retrieval features are extracted based on I-frame coding information in H.264. This paper proposes using a local mode histogram as the texture feature to match images and applying the residual coefficients to filter non-confident modes. The geometrical correspondence between two images is also considered. The experimental results show that the proposed method can substantially reduce computational and memory resource consumption, and provides similar performance compared with methods that extract features from decompressed images.     

Multiple exemplar-based facial image retrieval using independent component analysis.

In this paper, we design a content-based image retrieval system where multiple query examples can be used to indicate the need to retrieve not only images similar to the individual examples, but also those images which actually represent a combination of the content of query images. We propose a scheme for representing content of an image as a combination of features from multiple examples. This scheme is exploited for developing a multiple example-based retrieval engine. We have explored the use of machine learning techniques for generating the most appropriate feature combination scheme for a given class of images. The combination scheme can be used for developing purposive query engines for specialized image databases. Here, we have considered facial image databases. The effectiveness of the image retrieval system is experimentally demonstrated on different databases.     

Automatic image hanging protocol for chest radiographs in PACS.

Chest radiography is one of the most widely used techniques in diagnostic imaging. It comprises at least one-third of all diagnostic radiographic procedures in hospitals. However, in the picture archive and communication system, images are often stored with the projection and orientation unknown or mislabeled, which causes inefficiency for radiologists' interpretation. To address this problem, an automatic hanging protocol for chest radiographs is presented. The method targets the most effective region in a chest radiograph, and extracts a set of size-, rotation-, and translation-invariant features from it. Then, a well-trained classifier is used to recognize the projection. The orientation of the radiograph is later identified by locating the neck, heart, and abdomen positions in the radiographs. Initial experiments are performed on the radiographs collected from daily routine chest exams in hospitals and show promising results. Using the presented protocol, 98.2% of all cases could be hung correctly on projection view (without protocol, 62%), and 96.1% had correct orientation (without protocol, 75%). A workflow study on the protocol also demonstrates a significant improvement in efficiency for image display.     

X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words

In this study we present an efficient image categorization and retrieval system applied to medical image databases, in particular large radiograph archives. The methodology is based on local patch representation of the image content, using a "bag of visual words" approach. We explore the effects of various parameters on system performance, and show best results using dense sampling of simple features with spatial content, and a nonlinear kernel-based support vector machine (SVM) classifier. In a recent international competition the system was ranked first in discriminating orientation and body regions in X-ray images. In addition to organ-level discrimination, we show an application to pathology-level categorization of chest X-ray data, the most popular examination in radiology. The system discriminates between healthy and pathological cases, and is also shown to successfully identify specific pathologies in a set of chest radiographs taken from a routine hospital examination. This is a first step towards similarity-based categorization, which has a major clinical implications for computer-assisted diagnostics.     

Dictionary-based color image retrieval using multiset theory

Dictionaries have recently attracted a great deal of interest as a new powerful representation scheme that can describe the visual content of an image. Most existing approaches nevertheless, neglect dictionary statistics. In this work, we explore the linguistic and statistical properties of dictionaries in an image retrieval task, representing the dictionary as a multiset. This is extracted by means of the LZW data compressor which encodes the visual patterns of an image. For this reason the image is first quantized and then transformed into a 1D string of characters. Based on the multiset notion we also introduce the Normalized Multiset Distance (NMD), as a new dictionary-based dissimilarity measure which enables the user to retrieve images with similar content to a given query. Experimental results demonstrate a significant improvement in retrieval performance compared to related dictionary-based techniques or to several other image indexing methods that utilize classical low-level image features.     

Generalized manifold-ranking-based image retrieval.

In this paper, we propose a general transductive learning framework named generalized manifold-ranking-based image retrieval (gMRBIR) for image retrieval. Comparing with an existing transductive learning method named MRBIR [12], our method could work well whether or not the query image is in the database; thus, it is more applicable for real applications. Given a query image, gMRBIR first initializes a pseudo seed vector based on neighborhood relationship and then spread its scores via manifold ranking to all the unlabeled images in the database. Furthermore, in gMRBIR, we also make use of relevance feedback and active learning to refine the retrieval result so that it converges to the query concept as fast as possible. Systematic experiments on a general-purpose image database consisting of 5,000 Corel images demonstrate the superiority of gMRBIR over state-of-the-art techniques.     

Active learning methods for interactive image retrieval.

Active learning methods have been considered with increased interest in the statistical learning community. Initially developed within a classification framework, a lot of extensions are now being proposed to handle multimedia applications. This paper provides algorithms within a statistical framework to extend active learning for online content-based image retrieval (CBIR). The classification framework is presented with experiments to compare several powerful classification techniques in this information retrieval context. Focusing on interactive methods, active learning strategy is then described. The limitations of this approach for CBIR are emphasized before presenting our new active selection process RETIN. First, as any active method is sensitive to the boundary estimation between classes, the RETIN strategy carries out a boundary correction to make the retrieval process more robust. Second, the criterion of generalization error to optimize the active learning selection is modified to better represent the CBIR objective of database ranking. Third, a batch processing of images is proposed. Our strategy leads to a fast and efficient active learning scheme to retrieve sets of online images (query concept). Experiments on large databases show that the RETIN method performs well in comparison to several other active strategies.     

Structural image retrieval using automatic image annotation and region based inverted file

Image retrieval has lagged far behind text retrieval despite more than two decades of intensive research effort. Most of the research on image retrieval in the last two decades are on content based image retrieval or image retrieval based on low level features. Recent research in this area focuses on semantic image retrieval using automatic image annotation. Most semantic image retrieval techniques in literature, however, treat an image as a bag of features/words while ignore the structural or spatial information in the image. In this paper, we propose a structural image retrieval method based on automatic image annotation and region based inverted file. In the proposed system, regions in an image are treated the same way as keywords in a structural text document, semantic concepts are learnt from image data to label image regions as keywords and weight is assigned to each keyword according to spatial position and relationship. As the result, images are indexed and retrieved in the same way as structural document retrieval. Specifically, images are broken down to regions which are represented using colour, texture and shape features. Region features are then quantized to create visual dictionaries which are similar to monolingual dictionaries like English or Chinese dictionaries. In the next step, a semantic dictionary similar to a bilingual dictionary like the English–Chinese dictionary is learnt to mapping image regions to semantic concepts. Finally, images are then indexed and retrieved using a novel region based inverted file data structure. Results show the proposed method has significant advantage over the widely used Bayesian annotation models.     

Multitraining support vector machine for image retrieval.

Relevance feedback (RF) schemes based on support vector machines (SVMs) have been widely used in content-based image retrieval (CBIR). However, the performance of SVM-based RF approaches is often poor when the number of labeled feedback samples is small. This is mainly due to 1) the SVM classifier being unstable for small-size training sets because its optimal hyper plane is too sensitive to the training examples; and 2) the kernel method being ineffective because the feature dimension is much greater than the size of the training samples. In this paper, we develop a new machine learning technique, multitraining SVM (MTSVM), which combines the merits of the cotraining technique and a random sampling method in the feature space. Based on the proposed MTSVM algorithm, the above two problems can be mitigated. Experiments are carried out on a large image set of some 20,000 images, and the preliminary results demonstrate that the developed method consistently improves the performance over conventional SVM-based RFs in terms of precision and standard deviation, which are used to evaluate the effectiveness and robustness of a RF algorithm, respectively.