An endmember-based distance for content based hyperspectral image retrieval

We propose a specific content-based image retrieval (CBIR) system for hyperspectral images exploiting its rich spectral information. The CBIR image features are the endmember signatures obtained from the image data by endmember induction algorithms (EIAs). Endmembers correspond to the elementary materials in the scene, so that the pixel spectra can be decomposed into a linear combination of endmember signatures. EIA search for points in the high dimensional space of pixel spectra defining a convex polytope, often a simplex, covering the image data. This paper introduces a dissimilarity measure between hyperspectral images computed over the image induced endmembers, proving that it complies with the axioms of a distance. We provide a comparative discussion of dissimilarity functions, and quantitative evaluation of their relative performances on a large collection of synthetic hyperspectral images, and on a dataset extracted from a real hyperspectral image. Alternative dissimilarity functions considered are the Hausdorff distance and robust variations of it. We assess the CBIR performance sensitivity to changes in the distance between endmembers, the EIA employed, and some other conditions. The proposed hyperspectral image distance improves over the alternative dissimilarities in all quantitative performance measures. The visual results of the CBIR on the real image data demonstrate its usefulness for practical applications.     

Direct kernel biased discriminant analysis: a new content-based image retrieval relevance feedback algorithm

In recent years, a variety of relevance feedback (RF) schemes have been developed to improve the performance of content-based image retrieval (CBIR). Given user feedback information, the key to a RF scheme is how to select a subset of image features to construct a suitable dissimilarity measure. Among various RF schemes, biased discriminant analysis (BDA) based RF is one of the most promising. It is based on the observation that all positive samples are alike, while in general each negative sample is negative in its own way. However, to use BDA, the small sample size (SSS) problem is a big challenge, as users tend to give a small number of feedback samples. To explore solutions to this issue, this paper proposes a direct kernel BDA (DKBDA), which is less sensitive to SSS. An incremental DKBDA (IDKBDA) is also developed to speed up the analysis. Experimental results are reported on a real-world image collection to demonstrate that the proposed methods outperform the traditional kernel BDA (KBDA) and the support vector machine (SVM) based RF algorithms.     

An interactive approach for CBIR using a network of radial basis functions

An important requirement for constructing effective content-based image retrieval (CBIR) systems is accurate characterization of visual information. Conventional nonadaptive models, which are usually adopted for this task in simple CBIR systems, do not adequately capture all aspects of the characteristics of the human visual system. An effective way of addressing this problem is to adopt a "human-computer" interactive approach, where the users directly teach the system about what they regard as being significant image features and their own notions of image similarity. We propose a machine learning approach for this task, which allows users to directly modify query characteristics by specifying their attributes in the form of training examples. Specifically, we apply a radial-basis function (RBF) network for implementing an adaptive metric which progressively models the notion of image similarity through continual relevance feedback from users. Experimental results show that the proposed methods not only outperform conventional CBIR systems in terms of both accuracy and robustness, but also previously proposed interactive systems.     

基于前向神经网络的交互式图像检索系统

相关反馈技术是近年来基于内容图像检索中的研究重点,它有效地缩短了用户的高层语义概念同图像的底层视觉特征之间的差距,从而大大提高了系统的检索精度.本文对比了前向神经网络中的BP、FP和RBF三种网络学习算法;并在此基础上从机器学习的角度出发,分析了在图像检索中基于这三种网络的不同相关反馈技术.最后对今后的研究方向进行了展望.     

An effective technique for the content based image retrieval to reduce the semantic gap based on an optimal classifier technique

Content Based Image Retrieval (CBIR) systems use Relevance Feedback (RF) in order to improve the retrieval accuracy. Research focus has been shifted from designing sophisticated low-level feature extraction algorithms to reducing the “semantic gap” between the visual features and the richness of human semantics. In this paper, a novel system is proposed to enhance the gain of long-term relevance feedback. In the proposed system, the general CBIR involves two steps—ABC based training and image retrieval. First, the images other than the query image are pre-processed using median filter and gray scale transformation for removal of noise and resizing. Secondly, the features such as Color, Texture and shape of the image are extracted using Gabor Filter, Gray Level Co-occurrence Matrix and Hu-Moment shape feature techniques and also extract the static features like mean and standard deviation. The extracted features are clustered using k-means algorithm and each cluster are trained using ANN based ABC technique. A method using artificial bee colony (ABC) based artificial neural network (ANN) to update the weights assigned to features by accumulating the knowledge obtained from the user over iterations. Eventually, the comparative analysis performed using the commonly used methods namely precision and recall were clearly shown that the proposed system is suitable for the better CBIR and it can reduce the semantic gap than the conventional systems.     

Parallel heterogeneous CBIR system for efficient hyperspectral image retrieval using spectral mixture analysis

The purpose of content‐based image retrieval (CBIR) is to retrieve, from real data stored in a database, information that is relevant to a query. In remote sensing applications, the wealth of spectral information provided by latest‐generation (hyperspectral) instruments has quickly introduced the need for parallel CBIR systems able to effectively retrieve features of interest from ever‐growing data archives. To address this need, this paper develops a new parallel CBIR system that has been specifically designed to be run on heterogeneous networks of computers (HNOCs). These platforms have soon become a standard computing architecture in remote sensing missions due to the distributed nature of data repositories. The proposed heterogeneous system first extracts an image feature vector able to characterize image content with sub‐pixel precision using spectral mixture analysis concepts, and then uses the obtained feature as a search reference. The system is validated using a complex hyperspectral image database, and implemented on several networks of workstations and a Beowulf cluster at NASA's Goddard Space Flight Center. Our experimental results indicate that the proposed parallel system can efficiently retrieve hyperspectral images from complex image databases by efficiently adapting to the underlying parallel platform on which it is run, regardless of the heterogeneity in the compute nodes and communication links that form such parallel platform. Copyright © 2009 John Wiley & Sons, Ltd.     

Automatic machine interactions for content-based image retrieval using a self-organizing tree map architecture

In this paper, an unsupervised learning network is explored to incorporate a self-learning capability into image retrieval systems. Our proposal is a new attempt to automate recursive content-based image retrieval. The adoption of a self-organizing tree map (SOTM) is introduced, to minimize the user participation in an effort to automate interactive retrieval. The automatic learning mode has been applied to optimize the relevance feedback (RF) method and the single radial basis function-based RF method. In addition, a semiautomatic version is proposed to support retrieval with different user subjectivities. Image similarity is evaluated by a nonlinear model, which performs discrimination based on local analysis. Experimental results show robust and accurate performance by the proposed method, as compared with conventional noninteractive content-based image retrieval (CBIR) systems and user controlled interactive systems, when applied to image retrieval in compressed and uncompressed image databases.     

Relevance Feedback and Learning in Content-Based Image Search

A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.     

发掘相关反馈日志中关联信息的图像检索方法

相关反馈日志蕴含着丰富的对象语义关联信息,但大多数基于内容的图像检索(CBIR)方法却缺乏对它们的重用.提出一种发掘反馈日志中图像关联信息的自动化图像检索方法,将反馈事例中图像的共生现象视为一定上下文中的图像分类.检索时,结合CBIR的检索结果和多种上下文中的图像分类实例,借鉴HITS算法的思想从中提炼图像的本质性关联,获得综合内容和语义的图像检索结果.对6万幅Corel图像数据库的实验表明,该方法可以显著改善查全率和查准率,且检索结果能够更好地满足用户的语义检索需求.     

基于内容的图象检索技术

随着数字图象的日益增多,基于内容的图象检索已成为图象使用者和管理者迫切需要解决的问题,近年来,各国研究者纷纷加入该领域的研究.为了使人们对该领域现状有个概略了解,以推动该领域研究进一步开展,首先概括介绍了基于内容图象检索的产生、发展及其关键技术;然后介绍了特征提取(包括低层特征和语义特征)及其相似性计算、相关反馈等的原理及算法;最后指出了基于内容的图象检索技术与计算机视觉技术的区别所在,并对目前存在的问题和应着重的研究内容以及发展方向进行了分析.     

Long-Term Cross-Session Relevance Feedback Using Virtual Features

Relevance feedback (RF) is an iterative process, which refines the retrievals by utilizing the user's feedback on previously retrieved results. Traditional RF techniques solely use the short-term learning experience and do not exploit the knowledge created during cross sessions with multiple users. In this paper, we propose a novel RF framework, which facilitates the combination of short-term and long-term learning processes by integrating the traditional methods with a new technique called the virtual feature. The feedback history with all the users is digested by the system and is represented in a very efficient form as a virtual feature of the images. As such, the dissimilarity measure can dynamically be adapted, depending on the estimate of the semantic relevance derived from the virtual features. In addition, with a dynamic database, the user's subject concepts may transit from one to another. By monitoring the changes in retrieval performance, the proposed system can automatically adapt the concepts according to the new subject concepts. The experiments are conducted on a real image database. The results manifest that the proposed framework outperforms the traditional within-session and log-based long-term RF techniques.     

Content-based image retrieval using growing hierarchical self-organizing quadtree map

In this paper, a growing hierarchical self-organizing quadtree map (GHSOQM) is proposed and used for a content-based image retrieval (CBIR) system. The incorporation of GHSOQM in a CBIR system organizes images in a hierarchical structure. The retrieval time by GHSOQM is less than that by using direct image comparison using a flat structure. Furthermore, the ability of incremental learning enables GHSOQM to be a prospective neural-network-based approach for CBIR systems. We also propose feature matrices, image distance and relevance feedback for region-based images in the GHSOQM-based CBIR system. Experimental results strongly demonstrate the effectiveness of the proposed system.     

A novel relevance feedback method for CBIR

In this paper, we address the challenge about insufficiency of training set and limited feedback information in each relevance feedback (RF) round during the process of content based image retrieval (CBIR). We propose a novel active learning scheme to utilize the labeled and unlabeled images to build the initial Support Vector Machine (SVM) classifier for image retrieving. In our framework, two main components, a pseudo-label strategy and an improved active learning selection method, are included. Moreover, a feature subspace partition algorithm is proposed to model the retrieval target from users by the analysis from relevance labeled images. Experimental results demonstrate the superiority of the proposed method on a range of databases with respect to the retrieval accuracy.     

Generalized biased discriminant analysis for content-based image retrieval

Biased discriminant analysis (BDA) is one of the most promising relevance feedback (RF) approaches to deal with the feedback sample imbalance problem for content-based image retrieval (CBIR). However, the singular problem of the positive within-class scatter and the Gaussian distribution assumption for positive samples are two main obstacles impeding the performance of BDA RF for CBIR. To avoid both of these intrinsic problems in BDA, in this paper, we propose a novel algorithm called generalized BDA (GBDA) for CBIR. The GBDA algorithm avoids the singular problem by adopting the differential scatter discriminant criterion (DSDC) and handles the Gaussian distribution assumption by redesigning the between-class scatter with a nearest neighbor approach. To alleviate the overfitting problem, GBDA integrates the locality preserving principle; therefore, a smooth and locally consistent transform can also be learned. Extensive experiments show that GBDA can substantially outperform the original BDA, its variations, and related support-vector-machine-based RF algorithms.     

高光谱遥感图像的单形体分析方法

将n个波段的高光谱图像像元与n维空间里的散点联系起来，结合凸体几何中单形体概念研究高光谱遥感图像纯净像元提取方法，实现图像的地物精确分类识别及像元波谱分解。寻找高光谱遥感图像n维空间里的单形体并认知分析单形体是该研究方法的重要环节。通过MNF(minimum noise fraction)变换和PPI(pixel purity index)计算技术寻找到单形体，基于单形体进行像元分解分析单形体，并结合应用实例和SAM(spectral angle mapper)分类技术完成高光谱图像地物精确分类制图，验证了该研究方法的可操作性。该研究方法的优点在于不需要用户提供地物波谱信息，用于制图和波谱分解的终端单元可由图像本身得到，并由用户控制分类制图和波谱分解的详细程度。     

Information fusion between short term learning and long term learning in content based image retrieval systems

Content based image retrieval (CBIR) systems could provide more precise results by taking the user’s feedbacks into account. Two types of the relevance feedback learning paradigms are short term learning (STL) and long term learning (LTL). By using both STL and LTL, a collaborative CBIR system is proposed in this paper. The proposed system introduced three fusion methods: including fusion in retrieved images, fusion in ranks, and fusion in similarities to make cooperation between STL and LTL. The proposed fusion methods are examined in a CBIR system equipped with a proposed statistical semantic clustering (SSC) method of LTL. The SSC method works based on the concept of semantic categories of the images by clustering techniques and constructing a relevancy matrix between images and semantic categories. The results of the SSC method with the suggested fusion methods are compared with two state-of-the-art LTL methods, namely virtual feature based method and dynamic semantic clustering. Comparative results confirm the efficiency of the proposed method. Furthermore, experimental results demonstrate that for a unique LTL method, various fusion methods lead to different results.     

Interactive Search by Direct Manipulation of Dissimilarity Space

In this paper, we argue to learn dissimilarity for interactive search in content based image retrieval. In literature, dissimilarity is often learned via the feature space by feature selection, feature weighting or by adjusting the parameters of a function of the features. Other than existing techniques, we use feedback to adjust the dissimilarity space independent of feature space. This has the great advantage that it manipulates dissimilarity directly. To create a dissimilarity space, we use the method proposed by Pekalska and Duin, selecting a set of images called prototypes and computing distances to those prototypes for all images in the collection. After the user gives feedback, we apply active learning with a one-class support vector machine to decide the movement of images such that relevant images stay close together while irrelevant ones are pushed away (the work of Guo ). The dissimilarity space is then adjusted accordingly. Results on a Corel dataset of 10000 images and a TrecVid collection of 43907 keyframes show that our proposed approach is not only intuitive, it also significantly improves the retrieval performance.     

Probabilistic sequence clustering with spectral learning

In this paper, we derive two novel learning algorithms for time series clustering; namely for learning mixtures of Markov Models and mixtures of Hidden Markov Models. Mixture models are special latent variable models that require the usage of local search heuristics such as Expectation Maximization (EM) algorithm, that can only provide locally optimal solutions. In contrast, we make use of the spectral learning algorithms, recently popularized in the machine learning community. Under mild assumptions, spectral learning algorithms are able to estimate the parameters in latent variable models by solving systems of equations via eigendecompositions of matrices or tensors of observable moments. As such, spectral methods can be viewed as an instance of the method of moments for parameter estimation, an alternative to maximum likelihood. The popularity stems from the fact that these methods provide a computationally cheap and local optima free alternative to EM. We conduct classification experiments on human action sequences extracted from videos, clustering experiments on motion capture data and network traffic data to illustrate the viability of our approach. We conclude that the spectral methods are a practical and useful alternative in terms of computational effort and solution quality to standard iterative techniques such as EM in several sequence clustering applications.     

Neighborhood preserving regression for image retrieval

In content-based image retrieval (CBIR), relevance feedback has been proven to be a powerful tool for bridging the gap between low level visual features and high level semantic concepts. Traditionally, relevance feedback driven CBIR is often considered as a supervised learning problem where the user provided feedbacks are used to learn a distance metric or classification function. However, CBIR is intrinsically a semi-supervised learning problem in which the testing samples (images in the database) are present during the learning process. Moreover, when there are no sufficient feedbacks, these methods may suffer from the overfitting problem. In this paper, we propose a novel neighborhood preserving regression algorithm which makes efficient use of both labeled and unlabeled images. By using the unlabeled images, the geometrical structure of the image space can be incorporated into the learning system through a regularizer. Specifically, from all the functions which minimize the empirical loss on the labeled images, we select the one which best preserves the local neighborhood structure of the image space. In this way, our method can obtain a regression function which respects both semantic and geometrical structures of the image database. We present experimental evidence suggesting that our algorithm is able to use unlabeled data effectively for image retrieval.