A novel long-term learning algorithm for relevance feedback in content-based image retrieval

Relevance feedback has proven to be a powerful tool to bridge the semantic gap between low-level features and high-level human concepts in content-based image retrieval (CBIR). However, traditional short-term relevance feedback technologies are confined to using the current feedback record only. Log-based long-term learning captures the semantic relationships among images in a database by analyzing the historical relevance information to boost the retrieval performance effectively. In this paper, we propose an expanded-judging model to analyze the historical log data’s semantic information and to expand the feedback sample set from both positive and negative relevant information. The index table is used to facilitate the log analysis. The expanded-judging model is applied in image retrieval by combining with short-term relevance feedback algorithms. Experiments were carried out to evaluate the proposed algorithm based on the Corel image database. The promising experimental results validate the effectiveness of our proposed expanded-judging model.     

DRM: dynamic region matching for image retrieval using probabilistic fuzzy matching and boosting feature selection

This paper considers the semantic gap in content-based image retrieval from two aspects: (1) irrelevant visual contents (e.g. background) scatter the mapping from image to human perception; (2) unsupervised feature extraction and similarity ranking method can not accurately reveal users’ image perception. This paper proposes a novel region-based retrieval framework—dynamic region matching (DRM) to bridge the semantic gap. (1) To address the first issue, a probabilistic fuzzy region matching algorithm is adopted to retrieve and match images precisely at object level, which copes with the problem of inaccurate segmentation. (2) To address the second issue, a “FeatureBoost” algorithm is proposed to construct an effective “eigen” feature set in relevance feedback (RF) process. And the significance of each region is dynamically updated in RF learning to automatically capture users’ region of interest (ROI). (3) User’s retrieval purpose is predicted using a novel log-learning algorithm, which predicts users’ retrieval target in the feature space using the accumulated user operations. Extensive experiments have been conducted on Corel image database with over 10,000 images. The promising experimental results reveal the effectiveness of our scheme in bridging the semantic gap.     

Semantic content-based image retrieval: A comprehensive study

The complexity of multimedia contents is significantly increasing in the current digital world. This yields an exigent demand for developing highly effective retrieval systems to satisfy human needs. Recently, extensive research efforts have been presented and conducted in the field of content-based image retrieval (CBIR). The majority of these efforts have been concentrated on reducing the semantic gap that exists between low-level image features represented by digital machines and the profusion of high-level human perception used to perceive images. Based on the growing research in the recent years, this paper provides a comprehensive review on the state-of-the-art in the field of CBIR. Additionally, this study presents a detailed overview of the CBIR framework and improvements achieved; including image preprocessing, feature extraction and indexing, system learning, benchmarking datasets, similarity matching, relevance feedback, performance evaluation, and visualization. Finally, promising research trends, challenges, and our insights are provided to inspire further research efforts.     

CLUE: cluster-based retrieval of images by unsupervised learning.

In a typical content-based image retrieval (CBIR) system, target images (images in the database) are sorted by feature similarities with respect to the query. Similarities among target images are usually ignored. This paper introduces a new technique, cluster-based retrieval of images by unsupervised learning (CLUE), for improving user interaction with image retrieval systems by fully exploiting the similarity information. CLUE retrieves image clusters by applying a graph-theoretic clustering algorithm to a collection of images in the vicinity of the query. Clustering in CLUE is dynamic. In particular, clusters formed depend on which images are retrieved in response to the query. CLUE can be combined with any real-valued symmetric similarity measure (metric or nonmetric). Thus, it may be embedded in many current CBIR systems, including relevance feedback systems. The performance of an experimental image retrieval system using CLUE is evaluated on a database of around 60,000 images from COREL. Empirical results demonstrate improved performance compared with a CBIR system using the same image similarity measure. In addition, results on images returned by Google's Image Search reveal the potential of applying CLUE to real-world image data and integrating CLUE as a part of the interface for keyword-based image retrieval systems.     

A similarity learning approach to content-based image retrieval: application to digital mammography

In this paper, we describe an approach to content-based retrieval of medical images from a database, and provide a preliminary demonstration of our approach as applied to retrieval of digital mammograms. Content-based image retrieval (CBIR) refers to the retrieval of images from a database using information derived from the images themselves, rather than solely from accompanying text indices. In the medical-imaging context, the ultimate aim of CBIR is to provide radiologists with a diagnostic aid in the form of a display of relevant past cases, along with proven pathology and other suitable information. CBIR may also be useful as a training tool for medical students and residents. The goal of information retrieval is to recall from a database information that is relevant to the user's query. The most challenging aspect of CBIR is the definition of relevance (similarity), which is used to guide the retrieval machine. In this paper, we pursue a new approach, in which similarity is learned from training examples provided by human observers. Specifically, we explore the use of neural networks and support vector machines to predict the user's notion of similarity. Within this framework we propose using a hierarchal learning approach, which consists of a cascade of a binary classifier and a regression module to optimize retrieval effectiveness and efficiency. We also explore how to incorporate online human interaction to achieve relevance feedback in this learning framework. Our experiments are based on a database consisting of 76 mammograms, all of which contain clustered microcalcifications (MCs). Our goal is to retrieve mammogram images containing similar MC clusters to that in a query. The performance of the retrieval system is evaluated using precision-recall curves computed using a cross-validation procedure. Our experimental results demonstrate that: 1) the learning framework can accurately predict the perceptual similarity reported by human observers, thereby serving as a basis for CBIR; 2) the learning-based framework can significantly outperform a simple distance-based similarity metric; 3) the use of the hierarchical two-stage network can improve retrieval performance; and 4) relevance feedback can be effectively incorporated into this learning framework to achieve improvement in retrieval precision based on online interaction with users; and 5) the retrieved images by the network can have predicting value for the disease condition of the query.     

一种自适应提取最优特征维的相关反馈算法

本文提出一种新的相关反馈算法,该算法依据用户的反馈信息自适应选取用户最感兴趣的特征维用于图像检索,并结合正负反馈图像集的预处理,图像检索精确度得到较大提高。算法在500幅和4500幅两个图像库中做了实验,通过与RuiY特征内相关反馈算法的比较,验证了算法的高效性。     

Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces, and progressive learning

Research has been devoted in the past few years to relevance feedback as an effective solution to improve performance of content-based image retrieval (CBIR). In this paper, we propose a new feedback approach with progressive learning capability combined with a novel method for the feature subspace extraction. The proposed approach is based on a Bayesian classifier and treats positive and negative feedback examples with different strategies. Positive examples are used to estimate a Gaussian distribution that represents the desired images for a given query; while the negative examples are used to modify the ranking of the retrieved candidates. In addition, feature subspace is extracted and updated during the feedback process using a principal component analysis (PCA) technique and based on user's feedback. That is, in addition to reducing the dimensionality of feature spaces, a proper subspace for each type of features is obtained in the feedback process to further improve the retrieval accuracy. Experiments demonstrate that the proposed method increases the retrieval speed, reduces the required memory and improves the retrieval accuracy significantly.     

基于潜在语义分析和相关反馈的图像检索方法

为了解决传统的CBIR系统中存在的"语义鸿沟"问题,提出一种基于潜在语义索引技术(LSI)和相关反馈技术的图像检索方法.在进行图像检索时,先在HSV空间下提取颜色直方图作为底层视觉特征进行图像检索,然后引入潜在语义索引技术试图将底层特征赋予更高层次的语义含义;并且结合相关反馈技术,通过与用户交互进一步提高检索精度.实验...     

A new way for multidimensional medical data management: volume of interest (VOI)-based retrieval of medical images with visual and functional features.

The advances in digital medical imaging and storage in integrated databases are resulting in growing demands for efficient image retrieval and management. Content-based image retrieval (CBIR) refers to the retrieval of images from a database, using the visual features derived from the information in the image, and has become an attractive approach to managing large medical image archives. In conventional CBIR systems for medical images, images are often segmented into regions which are used to derive two-dimensional visual features for region-based queries. Although such approach has the advantage of including only relevant regions in the formulation of a query, medical images that are inherently multidimensional can potentially benefit from the multidimensional feature extraction which could open up new opportunities in visual feature extraction and retrieval. In this study, we present a volume of interest (VOI) based content-based retrieval of four-dimensional (three spatial and one temporal) dynamic PET images. By segmenting the images into VOIs consisting of functionally similar voxels (e.g., a tumor structure), multidimensional visual and functional features were extracted and used as region-based query features. A prototype VOI-based functional image retrieval system (VOI-FIRS) has been designed to demonstrate the proposed multidimensional feature extraction and retrieval. Experimental results show that the proposed system allows for the retrieval of related images that constitute similar visual and functional VOI features, and can find potential applications in medical data management, such as to aid in education, diagnosis, and statistical analysis.     

基于视觉感知的图像检索的研究

基于内容图像检索的一个突出问题是图像低层特征与高层语义之间存在的巨大鸿沟.针对相关反馈和感兴趣区检测在弥补语义鸿沟时存在主观性强、耗时的缺点,提出了视觉信息是一种客观反映图像高层语义的新特征,基于视觉信息进行图像检索可以有效减小语义鸿沟;并在总结视觉感知的研究进展和实现方法的基础上,给出了基于视觉感知的图像检索在感兴趣区检测、图像分割、相关反馈和个性化检索四个方面的研究思路.     

An efficient and effective region-based image retrieval framework

An image retrieval framework that integrates efficient region-based representation in terms of storage and complexity and effective on-line learning capability is proposed. The framework consists of methods for region-based image representation and comparison, indexing using modified inverted files, relevance feedback, and learning region weighting. By exploiting a vector quantization method, both compact and sparse (vector) region-based image representations are achieved. Using the compact representation, an indexing scheme similar to the inverted file technology and an image similarity measure based on Earth Mover's Distance are presented. Moreover, the vector representation facilitates a weighted query point movement algorithm and the compact representation enables a classification-based algorithm for relevance feedback. Based on users' feedback information, a region weighting strategy is also introduced to optimally weight the regions and enable the system to self-improve. Experimental results on a database of 10,000 general-purposed images demonstrate the efficiency and effectiveness of the proposed framework.     

Multitraining support vector machine for image retrieval.

Relevance feedback (RF) schemes based on support vector machines (SVMs) have been widely used in content-based image retrieval (CBIR). However, the performance of SVM-based RF approaches is often poor when the number of labeled feedback samples is small. This is mainly due to 1) the SVM classifier being unstable for small-size training sets because its optimal hyper plane is too sensitive to the training examples; and 2) the kernel method being ineffective because the feature dimension is much greater than the size of the training samples. In this paper, we develop a new machine learning technique, multitraining SVM (MTSVM), which combines the merits of the cotraining technique and a random sampling method in the feature space. Based on the proposed MTSVM algorithm, the above two problems can be mitigated. Experiments are carried out on a large image set of some 20,000 images, and the preliminary results demonstrate that the developed method consistently improves the performance over conventional SVM-based RFs in terms of precision and standard deviation, which are used to evaluate the effectiveness and robustness of a RF algorithm, respectively.     

Support Vector Machine Learning for Region‐Based Image Retrieval with Relevance Feedback

We present a relevance feedback approach based on multi‐class support vector machine (SVM) learning and cluster‐merging which can significantly improve the retrieval performance in region‐based image retrieval. Semantically relevant images may exhibit various visual characteristics and may be scattered in several classes in the feature space due to the semantic gap between low‐level features and high‐level semantics in the user's mind. To find the semantic classes through relevance feedback, the proposed method reduces the burden of completely re‐clustering the classes at iterations and classifies multiple classes. Experimental results show that the proposed method is more effective and efficient than the two‐class SVM and multi‐class relevance feedback methods.     

Hybrid textual-visual relevance learning for content-based image retrieval

Learning effective relevance measures plays a crucial role in improving the performance of content-based image retrieval (CBIR) systems. Despite extensive research efforts for decades, how to discover and incorporate semantic information of images still poses a formidable challenge to real-world CBIR systems. In this paper, we propose a novel hybrid textual-visual relevance learning method, which mines textual relevance from image tags and combines textual relevance and visual relevance for CBIR. To alleviate the sparsity and unreliability of tags, we first perform tag completion to fill the missing tags as well as correct noisy tags of images. Then, we capture users’ semantic cognition to images by representing each image as a probability distribution over the permutations of tags. Finally, instead of early fusion, a ranking aggregation strategy is adopted to sew up textual relevance and visual relevance seamlessly. Extensive experiments on two benchmark datasets well verified the promise of our approach.     

A new SVM-based relevance feedback image retrieval using probabilistic feature and weighted kernel function

Relevance feedback (RF) is an effective approach to bridge the gap between low-level visual features and high-level semantic meanings in content-based image retrieval (CBIR). The support vector machine (SVM) based RF mechanisms have been used in different fields of image retrieval, but they often treat all positive and negative feedback samples equally, which will inevitably degrade the effectiveness of SVM-based RF approaches for CBIR. In fact, positive and negative feedback samples, different positive feedback samples, and different negative feedback samples all always have distinct properties. Moreover, each feedback interaction process is usually tedious and time-consuming because of complex visual features, so if too many times of iteration of feedback are asked, users may be impatient to interact with the CBIR system. To overcome the above limitations, we propose a new SVM-based RF approach using probabilistic feature and weighted kernel function in this paper. Firstly, the probabilistic features of each image are extracted by using principal components analysis (PCA) and the adapted Gaussian mixture models (AGMM) based dimension reduction, and the similarity is computed by employing Kullback–Leibler divergence. Secondly, the positive feedback samples and negative feedback samples are marked, and all feedback samples’ weight values are computed by utilizing the samples-based Relief feature weighting. Finally, the SVM kernel function is modified dynamically according to the feedback samples’ weight values. Extensive simulations on large databases show that the proposed algorithm is significantly more effective than the state-of-the-art approaches.     

基于混合学习框架的SVM反馈算法研究

 基于支持向量机（SupportVectorMachine,SVM）理论的相关反馈技术是可有效提高图像检索性能的重要手段之一。然而,大多数SVM反馈算法普遍受到小样本问题的制约。本文综合了集成学习、半监督学习和主动学习三种方法的技术特点,提出一种混合学习框架下的SVM反馈算法。该算法在Boosting迭代过程中使用了未标记图像,以增加个体SVM之间的差异,从而获得高效的集成学习模型。同时,高效的集成学习模型更有利于寻找富有信息（most informative）图像,从而也提高了用户主动反馈的效率。实验结果及对比分析表明,混合学习策略可有效改进相关反馈的性能。     

一种融合图学习与区域显著性分析的图像检索算法

 为弥合图像低层视觉特征和高层语义之间的语义鸿沟,改善图像检索的效果,机器学习算法经常被引入到图像检索问题中.通常情况下,机器学习算法是与相关反馈机制相结合,通过用户的交互操作,标定出若干正反例图像,很自然地就可以将图像检索问题转化为模式识别中的分类问题.目前融合区域显著性分析的区域图像检索算法尚没有与机器学习算法相融合.本文结合图像区域显著性分析,并针对用户参与反馈的情况,分别提出了两种图像检索解决方案.其一,在没有用户反馈以及用户只反馈正例图像的情形下,将图像检索问题转化为直推式学习问题(Transductive Learning),改进已有的基于图的半监督学习算法,提出了融合区域显著性分析的层次化图表示(Hierarchical Graph Representation)方式,用以实现标记传播;其二,在用户同时反馈正反例图像的情形下,利用用户反馈得到的正反例图像构建相似性邻接矩阵,通过流形排序算法(Manifold-Ranking)学习出用户感兴趣的查询目标概念并用相应的特征向量集合表示,并据此查询图像库返回用户语义相关的图像集合.实验结果验证了这两种检索策略的有效性.     

流形学习和相关反馈相融合的医学图像检索

图像检索是医学图像辅助诊断的基础,为了提高医学图像检索的正确率,提出一种流形学习和相关反馈相融合的医学图像检索算法(LLE-MF)。首先根据方块编码的思想提取颜色分量的信息熵,并利用邻域灰度共生矩阵提取纹理特征;然后采用非线性流形学习对颜色和纹理特征进行组合、降维处理,并采用欧式距离相似度量模型对图像初步进行检索,最后最小二乘支持向量机对初步检索结果进行相关反馈,并进行仿真测试。结果表明,相对于其它医学检索算法,LLE-MF不仅提高了医学图像的检索准确率,同时提高了医学图像的检索效率,可以准确地找到用户所需的图像.