CLUE: cluster-based retrieval of images by unsupervised learning.

In a typical content-based image retrieval (CBIR) system, target images (images in the database) are sorted by feature similarities with respect to the query. Similarities among target images are usually ignored. This paper introduces a new technique, cluster-based retrieval of images by unsupervised learning (CLUE), for improving user interaction with image retrieval systems by fully exploiting the similarity information. CLUE retrieves image clusters by applying a graph-theoretic clustering algorithm to a collection of images in the vicinity of the query. Clustering in CLUE is dynamic. In particular, clusters formed depend on which images are retrieved in response to the query. CLUE can be combined with any real-valued symmetric similarity measure (metric or nonmetric). Thus, it may be embedded in many current CBIR systems, including relevance feedback systems. The performance of an experimental image retrieval system using CLUE is evaluated on a database of around 60,000 images from COREL. Empirical results demonstrate improved performance compared with a CBIR system using the same image similarity measure. In addition, results on images returned by Google's Image Search reveal the potential of applying CLUE to real-world image data and integrating CLUE as a part of the interface for keyword-based image retrieval systems.     

贝叶斯框架下基于区域的相关反馈算法

融合基于区域的图像表达方式和相关反馈技术能够有效地提高图像检索的性能。由于现有的方法没有充分考虑相同语义类内区域特征的分布情况,进而无法对该类的语义信息进行有效的描述,为此该文提出了贝叶斯框架下基于区域的相关反馈模型。在每轮相关反馈中,通过在线学习区域的贝叶斯分类器,同时根据最近邻最小错误估计原则确定分类器的可信度,可以可靠地建立图像相似性度量的概率模型。此外,在应用非参数密度估计技术来构造语义类的特征分布时,针对区域分割的不精确性,该文还考虑了区域特征空间的总体分布因素,进而对区域的后验分布进行更可靠地估计。最后的实验说明了该文方法的有效性。     

Support Vector Machine Learning for Region‐Based Image Retrieval with Relevance Feedback

We present a relevance feedback approach based on multi‐class support vector machine (SVM) learning and cluster‐merging which can significantly improve the retrieval performance in region‐based image retrieval. Semantically relevant images may exhibit various visual characteristics and may be scattered in several classes in the feature space due to the semantic gap between low‐level features and high‐level semantics in the user's mind. To find the semantic classes through relevance feedback, the proposed method reduces the burden of completely re‐clustering the classes at iterations and classifies multiple classes. Experimental results show that the proposed method is more effective and efficient than the two‐class SVM and multi‐class relevance feedback methods.     

Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces, and progressive learning

Research has been devoted in the past few years to relevance feedback as an effective solution to improve performance of content-based image retrieval (CBIR). In this paper, we propose a new feedback approach with progressive learning capability combined with a novel method for the feature subspace extraction. The proposed approach is based on a Bayesian classifier and treats positive and negative feedback examples with different strategies. Positive examples are used to estimate a Gaussian distribution that represents the desired images for a given query; while the negative examples are used to modify the ranking of the retrieved candidates. In addition, feature subspace is extracted and updated during the feedback process using a principal component analysis (PCA) technique and based on user's feedback. That is, in addition to reducing the dimensionality of feature spaces, a proper subspace for each type of features is obtained in the feedback process to further improve the retrieval accuracy. Experiments demonstrate that the proposed method increases the retrieval speed, reduces the required memory and improves the retrieval accuracy significantly.     

A framework for medical image retrieval using machine learning and statistical similarity matching techniques with relevance feedback.

A content-based image retrieval (CBIR) framework for diverse collection of medical images of different imaging modalities, anatomic regions with different orientations and biological systems is proposed. Organization of images in such a database (DB) is well defined with predefined semantic categories; hence, it can be useful for category-specific searching. The proposed framework consists of machine learning methods for image prefiltering, similarity matching using statistical distance measures, and a relevance feedback (RF) scheme. To narrow down the semantic gap and increase the retrieval efficiency, we investigate both supervised and unsupervised learning techniques to associate low-level global image features (e.g., color, texture, and edge) in the projected PCA-based eigenspace with their high-level semantic and visual categories. Specially, we explore the use of a probabilistic multiclass support vector machine (SVM) and fuzzy c-mean (FCM) clustering for categorization and prefiltering of images to reduce the search space. A category-specific statistical similarity matching is proposed in a finer level on the prefiltered images. To incorporate a better perception subjectivity, an RF mechanism is also added to update the query parameters dynamically and adjust the proposed matching functions. Experiments are based on a ground-truth DB consisting of 5000 diverse medical images of 20 predefined categories. Analysis of results based on cross-validation (CV) accuracy and precision-recall for image categorization and retrieval is reported. It demonstrates the improvement, effectiveness, and efficiency achieved by the proposed framework.     

A unified framework for image retrieval using keyword and visual features.

In this paper, a unified image retrieval framework based on both keyword annotations and visual features is proposed. In this framework, a set of statistical models are built based on visual features of a small set of manually labeled images to represent semantic concepts and used to propagate keywords to other unlabeled images. These models are updated periodically when more images implicitly labeled by users become available through relevance feedback. In this sense, the keyword models serve the function of accumulation and memorization of knowledge learned from user-provided relevance feedback. Furthermore, two sets of effective and efficient similarity measures and relevance feedback schemes are proposed for query by keyword scenario and query by image example scenario, respectively. Keyword models are combined with visual features in these schemes. In particular, a new, entropy-based active learning strategy is introduced to improve the efficiency of relevance feedback for query by keyword. Furthermore, a new algorithm is proposed to estimate the keyword features of the search concept for query by image example. It is shown to be more appropriate than two existing relevance feedback algorithms. Experimental results demonstrate the effectiveness of the proposed framework.     

DRM: dynamic region matching for image retrieval using probabilistic fuzzy matching and boosting feature selection

This paper considers the semantic gap in content-based image retrieval from two aspects: (1) irrelevant visual contents (e.g. background) scatter the mapping from image to human perception; (2) unsupervised feature extraction and similarity ranking method can not accurately reveal users’ image perception. This paper proposes a novel region-based retrieval framework—dynamic region matching (DRM) to bridge the semantic gap. (1) To address the first issue, a probabilistic fuzzy region matching algorithm is adopted to retrieve and match images precisely at object level, which copes with the problem of inaccurate segmentation. (2) To address the second issue, a “FeatureBoost” algorithm is proposed to construct an effective “eigen” feature set in relevance feedback (RF) process. And the significance of each region is dynamically updated in RF learning to automatically capture users’ region of interest (ROI). (3) User’s retrieval purpose is predicted using a novel log-learning algorithm, which predicts users’ retrieval target in the feature space using the accumulated user operations. Extensive experiments have been conducted on Corel image database with over 10,000 images. The promising experimental results reveal the effectiveness of our scheme in bridging the semantic gap.     

Evaluation of relevance feedback schemes in content-based in retrieval systems

Multimedia content modeling, i.e., identification of semantically meaningful entities, is an arduous task mainly due to the fact that (a) humans perceive the content using high-level concepts and (b) the subjectivity of human perception, which often interprets the same content in a different way at different times. For this reason, an efficient content management system has to be adapted to current user's information needs and preferences through an on-line learning strategy based on users’ interaction. One adaptive learning strategy is relevance feedback, originally developed in traditional text-based information retrieval systems. In this way, the user interacts with the system to provide information about the relevance of the content, which is then fed back to the system to update its performance. In this paper, we evaluate and investigate three main types of relevance feedback algorithms; the Euclidean, the query point movements and the correlation-based approaches. In the first case, we examine heuristic and optimal techniques which are based either on the weighted or the generalized Euclidean distance. In the second case, we survey single and multipoint query movement schemes. As far as the third type is concerned, two different ways for parametrizing the normalized cross-correlation similarity metric are proposed. The first scales only the elements of the query feature vector and called query-scaling strategy, while the second scales both the query and the selected samples (query-sample scaling strategy). All the examined algorithms are evaluated using both subjective and objective criteria. Subjective evaluation is performed by depicting the best retrieved images as response of the system to a user's query. Instead, objective evaluation is obtained using standard criteria, such as the precision–recall curve and the average normalized modified retrieval rank (ANMRR). Furthermore, a newly objective criterion, called average normalized similarity metric distance is introduced which exploits the difference among the actual and ideal similarity measure among all best retrievals. Discussions and comparisons of all the aforementioned relevance feedback algorithms are presented.     

Similarity-based online feature selection in content-based image retrieval.

Content-based image retrieval (CBIR) has been more and more important in the last decade, and the gap between high-level semantic concepts and low-level visual features hinders further performance improvement. The problem of online feature selection is critical to really bridge this gap. In this paper, we investigate online feature selection in the relevance feedback learning process to improve the retrieval performance of the region-based image retrieval system. Our contributions are mainly in three areas. 1) A novel feature selection criterion is proposed, which is based on the psychological similarity between the positive and negative training sets. 2) An effective online feature selection algorithm is implemented in a boosting manner to select the most representative features for the current query concept and combine classifiers constructed over the selected features to retrieve images. 3) To apply the proposed feature selection method in region-based image retrieval systems, we propose a novel region-based representation to describe images in a uniform feature space with real-valued fuzzy features. Our system is suitable for online relevance feedback learning in CBIR by meeting the three requirements: learning with small size training set, the intrinsic asymmetry property of training samples, and the fast response requirement. Extensive experiments, including comparisons with many state-of-the-arts, show the effectiveness of our algorithm in improving the retrieval performance and saving the processing time.     

An efficient and effective region-based image retrieval framework

An image retrieval framework that integrates efficient region-based representation in terms of storage and complexity and effective on-line learning capability is proposed. The framework consists of methods for region-based image representation and comparison, indexing using modified inverted files, relevance feedback, and learning region weighting. By exploiting a vector quantization method, both compact and sparse (vector) region-based image representations are achieved. Using the compact representation, an indexing scheme similar to the inverted file technology and an image similarity measure based on Earth Mover's Distance are presented. Moreover, the vector representation facilitates a weighted query point movement algorithm and the compact representation enables a classification-based algorithm for relevance feedback. Based on users' feedback information, a region weighting strategy is also introduced to optimally weight the regions and enable the system to self-improve. Experimental results on a database of 10,000 general-purposed images demonstrate the efficiency and effectiveness of the proposed framework.     

A novel long-term learning algorithm for relevance feedback in content-based image retrieval

Relevance feedback has proven to be a powerful tool to bridge the semantic gap between low-level features and high-level human concepts in content-based image retrieval (CBIR). However, traditional short-term relevance feedback technologies are confined to using the current feedback record only. Log-based long-term learning captures the semantic relationships among images in a database by analyzing the historical relevance information to boost the retrieval performance effectively. In this paper, we propose an expanded-judging model to analyze the historical log data’s semantic information and to expand the feedback sample set from both positive and negative relevant information. The index table is used to facilitate the log analysis. The expanded-judging model is applied in image retrieval by combining with short-term relevance feedback algorithms. Experiments were carried out to evaluate the proposed algorithm based on the Corel image database. The promising experimental results validate the effectiveness of our proposed expanded-judging model.     

基于教与学优化算法的相关反馈图像检索

为提高基于内容的图像检索的检索性能和检索速度,克服低层视觉特征与高层语义概念间的“语义鸿沟”,提出一种基于教与学优化的图像检索相关反馈算法（TLBO-RF）.结合图像检索问题的特殊性和粒子群优化算法的优点,对TLBO算法中个体的更新机制进行了改进,通过将相关图像集的中心作为教师以及引入学员最好学习状态Pbest,使之朝用户感兴趣的相关图像区域快速收敛.将该算法与目前效果最好的两种基于进化算法的相关反馈技术在两套标准图像测试集上进行对比,结果表明本文算法相较于另外两种算法具有明显的优势,不仅提高了图像检索性能,同时也加快了图像检索速度,更好地满足了用户的检索要求.     

A similarity learning approach to content-based image retrieval: application to digital mammography

In this paper, we describe an approach to content-based retrieval of medical images from a database, and provide a preliminary demonstration of our approach as applied to retrieval of digital mammograms. Content-based image retrieval (CBIR) refers to the retrieval of images from a database using information derived from the images themselves, rather than solely from accompanying text indices. In the medical-imaging context, the ultimate aim of CBIR is to provide radiologists with a diagnostic aid in the form of a display of relevant past cases, along with proven pathology and other suitable information. CBIR may also be useful as a training tool for medical students and residents. The goal of information retrieval is to recall from a database information that is relevant to the user's query. The most challenging aspect of CBIR is the definition of relevance (similarity), which is used to guide the retrieval machine. In this paper, we pursue a new approach, in which similarity is learned from training examples provided by human observers. Specifically, we explore the use of neural networks and support vector machines to predict the user's notion of similarity. Within this framework we propose using a hierarchal learning approach, which consists of a cascade of a binary classifier and a regression module to optimize retrieval effectiveness and efficiency. We also explore how to incorporate online human interaction to achieve relevance feedback in this learning framework. Our experiments are based on a database consisting of 76 mammograms, all of which contain clustered microcalcifications (MCs). Our goal is to retrieve mammogram images containing similar MC clusters to that in a query. The performance of the retrieval system is evaluated using precision-recall curves computed using a cross-validation procedure. Our experimental results demonstrate that: 1) the learning framework can accurately predict the perceptual similarity reported by human observers, thereby serving as a basis for CBIR; 2) the learning-based framework can significantly outperform a simple distance-based similarity metric; 3) the use of the hierarchical two-stage network can improve retrieval performance; and 4) relevance feedback can be effectively incorporated into this learning framework to achieve improvement in retrieval precision based on online interaction with users; and 5) the retrieved images by the network can have predicting value for the disease condition of the query.     

A Unified Relevance Feedback Framework for Web Image Retrieval

Although relevance feedback (RF) has been extensively studied in the content-based image retrieval community, no commercial Web image search engines support RF because of scalability, efficiency, and effectiveness issues. In this paper, we propose a unified relevance feedback framework for Web image retrieval. Our framework shows advantage over traditional RF mechanisms in the following three aspects. First, during the RF process, both textual feature and visual feature are used in a sequential way. To seamlessly combine textual feature-based RF and visual feature-based RF, a query concept-dependent fusion strategy is automatically learned. Second, the textual feature-based RF mechanism employs an effective search result clustering (SRC) algorithm to obtain salient phrases, based on which we could construct an accurate and low-dimensional textual space for the resulting Web images. Thus, we could integrate RF into Web image retrieval in a practical way. Last, a new user interface (UI) is proposed to support implicit RF. On the one hand, unlike traditional RF UI which enforces users to make explicit judgment on the results, the new UI regards the users' click-through data as implicit relevance feedback in order to release burden from the users. On the other hand, unlike traditional RF UI which hardily substitutes subsequent results for previous ones, a recommendation scheme is used to help the users better understand the feedback process and to mitigate the possible waiting caused by RF. Experimental results on a database consisting of nearly three million Web images show that the proposed framework is wieldy, scalable, and effective.      

The Bayesian image retrieval system, PicHunter: theory,implementation, and psychophysical experiments

Presents the theory, design principles, implementation and performance results of PicHunter, a prototype content-based image retrieval (CBIR) system. In addition, this document presents the rationale, design and results of psychophysical experiments that were conducted to address some key issues that arose during PicHunter's development. The PicHunter project makes four primary contributions to research on CBIR. First, PicHunter represents a simple instance of a general Bayesian framework which we describe for using relevance feedback to direct a search. With an explicit model of what users would do, given the target image they want, PicHunter uses Bayes's rule to predict the target they want, given their actions. This is done via a probability distribution over possible image targets, rather than by refining a query. Second, an entropy-minimizing display algorithm is described that attempts to maximize the information obtained from a user at each iteration of the search. Third, PicHunter makes use of hidden annotation rather than a possibly inaccurate/inconsistent annotation structure that the user must learn and make queries in. Finally, PicHunter introduces two experimental paradigms to quantitatively evaluate the performance of the system, and psychophysical experiments are presented that support the theoretical claims.     

Towards Optimal Indexing for Relevance Feedback in Large Image Databases$^+$

Motivated by the need to efficiently leverage user relevance feedback in content-based retrieval from image databases, we propose a fast, clustering-based indexing technique for exact nearest-neighbor search that adapts to the Mahalanobis distance with a varying weight matrix. We derive a basic property of point-to-hyperplane Mahalanobis distance, which enables efficient recalculation of such distances as the Mahalanobis weight matrix is varied. This property is exploited to recalculate bounds on query-cluster distances via projection on known separating hyperplanes (available from the underlying clustering procedure), to effectively eliminate noncompetitive clusters from the search and to retrieve clusters in increasing order of (the appropriate) distance from the query. We compare performance with an existing variant of VA-File indexing designed for relevance feedback, and observe considerable gains.      

基于本体与上下文感知矩阵的查询扩展算法

针对物理世界的信息查找在过去几年间也受到广泛关注,但是迄今还缺乏深入的研究.目前针对Web信息空间的搜索算法不适合普适空间内的信息查询,原因有二:面向物理实体查询的支撑技术,如嵌入式设备和无线通信,与传统Web信息搜索不同:物理实体相关的信息与Web网页不同,表现在元数据、信息动态性等方面.同时,由于用户查询用词与文档...     

A Dynamic User Concept Pattern Learning Framework for Content-Based Image Retrieval

A rapid increase in the amount of image data and the inefficiency of traditional text-based image retrieval systems have served to make content-based image retrieval an active research field. It is crucial to effectively discover users' concept patterns through an acquired understanding of the subjective role played by humans in the retrieval process for such systems. A learning and retrieval framework is used to achieve this. It seamlessly incorporates multiple instance learning for relevant feedback to discover users concept patterns-especially in the region of greatest user interest. It also maps the local feature vector of that region to the high-level concept pattern. This underlying mapping can be progressively discovered through feedback and learning. The user guides the retrieval systems learning process using his/her focus of attention. Retrieval performance is tested to establish the feasibility and effectiveness of the proposed learning and retrieval framework     

流形学习和相关反馈相融合的医学图像检索

图像检索是医学图像辅助诊断的基础,为了提高医学图像检索的正确率,提出一种流形学习和相关反馈相融合的医学图像检索算法(LLE-MF)。首先根据方块编码的思想提取颜色分量的信息熵,并利用邻域灰度共生矩阵提取纹理特征;然后采用非线性流形学习对颜色和纹理特征进行组合、降维处理,并采用欧式距离相似度量模型对图像初步进行检索,最后最小二乘支持向量机对初步检索结果进行相关反馈,并进行仿真测试。结果表明,相对于其它医学检索算法,LLE-MF不仅提高了医学图像的检索准确率,同时提高了医学图像的检索效率,可以准确地找到用户所需的图像.     

Combining positive and negative examples in relevance feedback for content-based image retrieval

In this paper, we address some issues related to the combination of positive and negative examples to improve the efficiency of image retrieval. We start by analyzing the relevance of the negative example and how it can be interpreted and utilized to mitigate certain problems in image retrieval, such as noise, miss, the page zero problem and feature selection. Then we propose a new relevance feedback approach that uses the positive example (PE) to perform generalization and the negative example (NE) to perform specialization. In this approach, a query containing both PE and NE is processed in two steps. The first step considers the PE alone, in order to reduce the set of images participating in retrieval to a more homogeneous subset. Then, the second step considers both PE and NE and acts on the images retained in the first step. Mathematically, relevance feedback is formulated as an optimization of the intra and inter variances of the PE and NE. The proposed relevance feedback algorithm was implemented in our image retrieval system, which we tested on a collection of more than 10,000 images. The experimental results show how the NE as considered in our model can contribute in improving the relevance of the images retrieved.