Incorporating multiple distance spaces in optimum-path forest classification to improve feedback-based learning

In content-based image retrieval (CBIR) using feedback-based learning, the user marks the relevance of returned images and the system learns how to return more relevant images in a next iteration. In this learning process, image comparison may be based on distinct distance spaces due to multiple visual content representations. This work improves the retrieval process by incorporating multiple distance spaces in a recent method based on optimum-path forest (OPF) classification. For a given training set with relevant and irrelevant images, an optimization algorithm finds the best distance function to compare images as a combination of their distances according to different representations. Two optimization techniques are evaluated: a multi-scale parameter search (MSPS), never used before for CBIR, and a genetic programming (GP) algorithm. The combined distance function is used to project an OPF classifier and to rank images classified as relevant for the next iteration. The ranking process takes into account relevant and irrelevant representatives, previously found by the OPF classifier. Experiments show the advantages in effectiveness of the proposed approach with both optimization techniques over the same approach with single distance space and over another state-of-the-art method based on multiple distance spaces.     

An image fusion framework using morphology and sparse representation

Image fusion is the process which aims to integrate the relevant and complementary information from a set of images into a single comprehensive image. Sparse representation (SR) is a powerful technique used in a wide variety of applications like denoising, compression and fusion. Building a compact and informative dictionary is the principal challenge in these applications. Hence, we propose a supervised classification based learning technique for the fusion algorithm. As an initial step, each patch of the training data set is pre-classified based on their gradient dominant direction. Then, a dictionary is learned using K-SVD algorithm. With this universal dictionary, sparse coefficients are estimated using greedy OMP algorithm to represent the given set of source images in the dominant direction. Finally, the Euclidean norm is used as a distance measure to reconstruct the fused image. Experimental results on different types of source images demonstrate the effectiveness of the proposed algorithm with conventional methods in terms of visual and quantitative evaluations.

     

Social image retrieval based on topic diversity

Image search re-ranking is one of the most important approaches to enhance the text-based image search results. Extensive efforts have been dedicated to improve the accuracy and diversity of tag-based image retrieval. However, how to make the top-ranked results relevant and diverse is still a challenging problem. In this paper, we propose a novel method to diversify the retrieval results by latent topic analysis. We first employ NMF (Non-negative Matrix Factorization) Lee and Seung (Nature 401(6755):788–791, 1999) to estimate the initial relevance score to the query q. Then, the initial relevance score is fed into an adaptive multi-feature fusion model to learn the final relevance score. Next, the diversification process is conducted. We group all the images by semantic clustering and estimate the topic distribution of each cluster by topic analysis. The clusters are ranked based on the topic distribution vector and the final retrieval image list is obtained by a greedy selection mechanism based on the estimated relevances. Experimental results on the NUS-Wide dataset show the effectiveness of the proposed approach.

     

Leveraging community metadata for multimodal image?ranking

Searching for relevant images given a query term is an important task in nowadays large-scale community databases. The image ranking approach presented in this work represents an image collection as a graph that is built using a multimodal similarity measure based on visual features and user tags. We perform a random walk on this graph to find the most common images. Further we discuss several scalability issues of the proposed approach and show how in this framework queries can be answered fast. Experimental results validate the effectiveness of the presented algorithm.     

Color indexing for efficient image retrieval

Content based image retrieval is an active area of research. Many approaches have been proposed to retrieve images based on matching of some features derived from the image content. Color is an important feature of image content. The problem with many traditional matching-based retrieval methods is that the search time for retrieving similar images for a given query image increases linearly with the size of the image database. We present an efficient color indexing scheme for similarity-based retrieval which has a search time that increases logarithmically with the database size.In our approach, the color features are extracted automatically using a color clustering algorithm. Then the cluster centroids are used as representatives of the images in 3-dimensional color space and are indexed using a spatial indexing method that usesR-tree. The worst case search time complexity of this approach isOn _q log(N^* n_avg)), whereN is the number of images in the database, andn _q andn _avg are the number of colors in the query image and the average number of colors per image in the database respectively. We present the experimental results for the proposed approach on two databases consisting of 337 Trademark images and 200 Flag images.     

Alternating Feature Spaces in Relevance Feedback

Image retrieval using relevance feedback can be treated as a two-class learning and classification process. The user-labelled relevant and irrelevant images are regarded as positive and negative training samples, based on which a classifier is trained dynamically. Then the classifier in turn classifies all images in the database. In practice, the number of training samples is very small because the users are often impatient. On the other hand, the positive samples usually are not representative since they are the nearest ones to the query and thus less informative. The insufficiency of training samples both in quantities and varieties constrains the generalization ability of the classifier significantly. In this paper, we propose a novel relevance feedback approach, which aims to collect more representative samples and hence improve the performance of classifier. Image labeling and classifier training are conducted in two complementary image feature spaces. Since the samples distribute differently in two spaces, the positive samples may be more informative in one feature space than in another. The two complementary feature spaces are alternated iteratively during the feedback process. To choose appropriate complementary feature spaces, we present two methods to measure the complementarities between two feature spaces quantitatively. Our experimental result on 10,000 images indicates that the proposed feedback approach significantly improves image retrieval performance.     

Improving keyword based web image search with visual feature distribution and term expansion

This paper discusses techniques for improving the performance of keyword-based web image queries. Firstly, a web page is segmented into several text blocks based on semantic cohesion. The text blocks which contain web images are taken as the associated texts of corresponding images and TF*IDF model is initially used to index those web images. Then, for each keyword, both relevant web image set and irrelevant web image set are selected according to their TF*IDF values. And visual feature distributions of both positive image and negative image are modeled using Gaussian Mixture Model. An image’s relevance to the keyword with respect to visual feature is thus defined as the ratio of positive distribution density over negative distribution density. We combine the text-based relevance model with visual feature relevance model to improve the performance. Thirdly, a query expansion model is used to improve the performance further. Expansion terms are selected according to their cooccurrences with the query terms in the top-relevant set of the original query. Our experiments show that our approach yield significant improvement over the traditional keyword based query model.     

基于改进的排序学习的图片检索算法研究

图片检索是图片共享社会网络中的重要研究内容之一。传统的图片检索方法往往通过对用户输入的关键字和图片的文本描述加以匹配来进行图片检索。由于文本信息存在歧义性,图片的文本描述十分困难,因此检索结果的准确性低。为了提高图片检索的准确性,提出了基于排序学习的图片检索方法。将每幅图片通过多种特征描述符进行描述,当用户的输入为图片时,通过对比查询图片和图片库中图片的相似性进行图片检索。采用支持向量机和关联规则两种学习方法对特征描述符的权重组合进行学习,并提出了相应的学习算法。实验表明,提出的基于学习的图片检索方法与相关图片检索方法相比具有更高的准确性。此外,应用支持向量机和关联规则两种方法对分类函数进行学习时,由于两种算法通过相同的数据实例对图片描述符的权重进行学习,因此得到的结果是相关的。     

Towards Data-Adaptive and User-Adaptive Image Retrieval by Peer Indexing

Adaptation to the characteristics of specific images and the preferences of individual users is critical to the success of an image retrieval system but insufficiently addressed by the existing approaches. In this paper, we propose an elegant and effective approach to data-adaptive and user-adaptive image retrieval based on the idea of peer indexing—describing an image through semantically relevant peer images. Specifically, we associate each image with a two-level peer index that models the “data characteristics” of the image as well as the “user characteristics” of individual users with respect to this image. Based on two-level image peer indexes, a set of retrieval parameters including query vectors and similarity metric are optimized towards both data and user characteristics by applying the pseudo feedback strategy. A cooperative framework is proposed under which peer indexes and image visual features are integrated to facilitate data- and user-adaptive image retrieval. Simulation experiments conducted on real-world images have verified the effectiveness of our approach in a relatively restricted setting.     

An object-based image retrieval system for digital libraries

A novel approach to clustering for image segmentation and a new object-based image retrieval method are proposed. The clustering is achieved using the Fisher discriminant as an objective function. The objective function is improved by adding a spatial constraint that encourages neighboring pixels to take on the same class label. A six-dimensional feature vector is used for clustering by way of the combination of color and busyness features for each pixel. After clustering, the dominant segments in each class are chosen based on area and used to extract features for image retrieval. The color content is represented using a histogram, and Haar wavelets are used to represent the texture feature of each segment. The image retrieval is segment-based; the user can select a query segment to perform the retrieval and assign weights to the image features. The distance between two images is calculated using the distance between features of the constituent segments. Each image is ranked based on this distance with respect to the query image segment. The algorithm is applied to a pilot database of natural images and is shown to improve upon the conventional classification and retrieval methods. The proposed segmentation leads to a higher number of relevant images retrieved, 83.5% on average compared to 72.8 and 68.7% for the k-means clustering and the global retrieval methods, respectively.     

Look‐a‐Like: A Fast Content‐Based Image Retrieval Approach Using a Hierarchically Nested Dynamically Evolving Image Clouds and Recursive Local Data Density

The need to find related images from big data streams is shared by many professionals, such as architects, engineers, designers, journalist, and ordinary people. Users need to quickly find the relevant images from data streams generated from a variety of domains. The challenges in image retrieval are widely recognized, and the research aiming to address them led to the area of content‐based image retrieval becoming a “hot” area. In this paper, we propose a novel computationally efficient approach, which provides a high visual quality result based on the use of local recursive density estimation between a given query image of interest and data clouds/clusters which have hierarchical dynamically nested evolving structure. The proposed approach makes use of a combination of multiple features. The results on a data set of 65,000 images organized in two layers of a hierarchy demonstrate its computational efficiency. Moreover, the proposed Look‐a‐like approach is self‐evolving and updating adding new images by crawling and from the queries made.     

Content based image retrieval via a transductive model

Content based image retrieval plays an important role in the management of a large image database. However, the results of state-of-the-art image retrieval approaches are not so satisfactory for the well-known gap between visual features and semantic concepts. Therefore, a novel transductive learning scheme named random walk with restart based method (RWRM) is proposed, consisting of three major components: pre-filtering processing, relevance score calculation, and candidate ranking refinement. Firstly, to deal with the problem of large computation cost involved in a large image database, a pre-filtering processing is utilized to filter out the most irrelevant images while keeping the most relevant images according to the results of a manifold ranking algorithm. Secondly, the relevance between a query image and the remaining images are obtained with respect to the probability density estimation. Finally, a transductive learning model, namely a random walk with restart model, is utilized to refine the ranking taking into account both the pairwise information of unlabeled images and the relevance scores between query image and unlabeled images. Experiments conducted on a typical Corel dataset demonstrate the effectiveness of the proposed scheme.     

A kernel-based framework for image collection exploration

While search engines have been a successful tool to search text information, image search systems still face challenges. The keyword-based query paradigm used to search in image collection systems, which has been successful in text retrieval, may not be useful in scenarios where the user does not have the precise way to express a visual query. Image collection exploration is a new paradigm where users interact with the image collection to discover useful and relevant pictures. This paper proposes a framework for the construction of an image collection exploration system based on kernel methods, which offers a mathematically strong basis to address each stage of an image collection exploration system: image representation, summarization, visualization and interaction. In particular, our approach emphasizes a semantic representation of images using kernel functions, which can be seamlessly harnessed across all system components. Experiments were conducted with real users to verify the effectiveness and efficiency of the proposed strategy.     

Query processing issues in region-based image databases

Many modern image database systems adopt a region-based paradigm, in which images are segmented into homogeneous regions in order to improve the retrieval accuracy. With respect to the case where images are dealt with as a whole, this leads to some peculiar query processing issues that have not been investigated so far in an integrated way. Thus, it is currently hard to understand how the different alternatives for implementing the region-based image retrieval model might impact on performance. In this paper, we analyze in detail such issues, in particular the type of matching between regions (either one-to-one or many-to-many). Then, we propose a novel ranking model, based on the concept of Skyline, as an alternative to the usual one based on aggregation functions and k-Nearest Neighbors queries. We also discuss how different query types can be efficiently supported. For all the considered scenarios we detail efficient index-based algorithms that are provably correct. Extensive experimental analysis shows, among other things, that: (1) the 1–1 matching type has to be preferred to the N–M one in terms of efficiency, whereas the two have comparable effectiveness, (2) indexing regions rather than images performs much better, and (3) the novel Skyline ranking model is consistently the most efficient one, even if this sometimes comes at the price of a reduced effectiveness.     

Web image retrieval using majority-based ranking approach

Web image retrieval has characteristics different from typical content-based image retrieval; web images have associated textual cues. However, a web image retrieval system often yields undesirable results, because it uses limited text information such as surrounding text, URLs, and image filenames. In this paper, we propose a new approach to retrieval, which uses the image content of retrieved results without relying on assistance from the user. Our basic hypothesis is that more popular images have a higher probability of being the ones that the user wishes to retrieve. According to this hypothesis, we propose a retrieval approach that is based on a majority of the images under consideration. We define four methods for finding the visual features of majority of images; (1) majority-first method, (2) centroid-of-all method, (3) centroid-of-top K method, and (4) centroid-of-largest-cluster method. In addition, we implement a graph/picture classifier for improving the effectiveness of web image retrieval. We evaluate the retrieval effectiveness of both our methods and conventional ones by using precision and recall graphs. Experimental results show that the proposed methods are more effective than conventional keyword-based retrieval methods.     

Patch-based active learning (PTAL) for spectral-spatial classification on hyperspectral data

Active learning (AL) has been shown to be a useful approach to improving the efficiency of the classification process for remote-sensing imagery. Current AL methods are essentially based on pixel-wise classification. In this paper, a new patch-based active learning (PTAL) framework is proposed for spectral-spatial classification on hyperspectral remote-sensing data. The method consists of two major steps. In the initialization stage, the original hyperspectral images are partitioned into overlapping patches. Then, for each patch, the spectral and spatial information as well as the label are extracted. A small set of patches is randomly selected from the data set for annotation, then a patch-based support vector machine (PTSVM) classifier is initially trained with these patches. In the second stage (close-loop stage of query and retraining), the trained PTSVM classifier is combined with one of three query methods, which are margin sampling (MS), entropy query-by-bagging (EQB), and multi-class level uncertainty (MCLU), and is subsequently employed to query the most informative samples from the candidate pool comprising the rest of the patches from the data set. The query selection cycle enables the PTSVM model to select the most informative queries for human annotation. Then, these informative queries are added to the training set. This process runs iteratively until a stopping criterion is met. Finally, the trained PTSVM is employed to patch classification. In order to compare this to pixel-based active learning (PXAL) models, the prediction label of a patch by PTSVM is transformed into a pixel-wise label of a pixel predictor to get the classification maps. Experimental results show better performance of the proposed PTAL methods on classification accuracy and computational time on three different hyperspectral data sets as compared with PXAL methods.     

Retrieval of clothing images based on relevance feedback with focus on collar designs

The content-based image retrieval methods are developed to help people find what they desire based on preferred images instead of linguistic information. This paper focuses on capturing the image features representing details of the collar designs, which is important for people to choose clothing. The quality of the feature extraction methods is important for the queries. This paper presents several new methods for the collar-design feature extraction. A prototype of clothing image retrieval system based on relevance feedback approach and optimum-path forest algorithm is also developed to improve the query results and allows users to find clothing image of more preferred design. A series of experiments are conducted to test the qualities of the feature extraction methods and validate the effectiveness and efficiency of the RF-OPF prototype from multiple aspects. The evaluation scores of initial query results are used to test the qualities of the feature extraction methods. The average scores of all RF steps, the average numbers of RF iterations taken before achieving desired results and the score transition of RF iterations are used to validate the effectiveness and efficiency of the proposed RF-OPF prototype.