An Efficient Deep Learning-based Content-based Image Retrieval Framework

The use of massive image databases has increased drastically over the few years due to evolution of multimedia technology. Image retrieval has become one of the vital tools in image processing applications. Content-Based Image Retrieval (CBIR) has been widely used in varied applications. But, the results produced by the usage of a single image feature are not satisfactory. So, multiple image features are used very often for attaining better results. But, fast and effective searching for relevant images from a database becomes a challenging task. In the previous existing system, the CBIR has used the combined feature extraction technique using color auto-correlogram, Rotation-Invariant Uniform Local Binary Patterns (RULBP) and local energy. However, the existing system does not provide significant results in terms of recall and precision. Also, the computational complexity is higher for the existing CBIR systems. In order to handle the above mentioned issues, the Gray Level Co-occurrence Matrix (GLCM) with Deep Learning based Enhanced Convolution Neural Network (DLECNN) is proposed in this work. The proposed system framework includes noise reduction using histogram equalization, feature extraction using GLCM, similarity matching computation using Hierarchal and Fuzzy c- Means (HFCM) algorithm and the image retrieval using DLECNN algorithm. The histogram equalization has been used for computing the image enhancement. This enhanced image has a uniform histogram. Then, the GLCM method has been used to extract the features such as shape, texture, colour, annotations and keywords. The HFCM similarity measure is used for computing the query image vector's similarity index with every database images. For enhancing the performance of this image retrieval approach, the DLECNN algorithm is proposed to retrieve more accurate features of the image. The proposed GLCM+DLECNN algorithm provides better results associated with high accuracy, precision, recall, f-measure and lesser complexity. From the experimental results, it is clearly observed that the proposed system provides efficient image retrieval for the given query image.     

基于内容图像检索中的特征性能评价

在基于内容的图像检索中，不同图像特征反映了图像各个侧面的内在特性，因此，在使用图像特征进行检索时存在多种相似性度量方法．特征以及特征间相似性度量方法的选取是当前CBIR研究的一个重要课题．评估了CBIR系统中使用的图像特征在不同相似性度量方法下及多种特征在不同图像库上的检索性能，为CBIR系统的设计和实现提供一定的依据．通过实验发现，图像特征的检索性能不仅同相似性度量方法有关系，同时与图像库也有密切的关系．     

利用二部图匹配进行图像相似性度量

基于内容图像检索是多媒体信息检索领域研究的热点，而现有的算法和系统离成熟的应用还相距甚远，其检索效率和准确性都相当低。提高基于内容图像检索性能的关键在于实现对图像的对象级访问，但是已有的很多的基于区域的图像检索算法和系统都没有考虑多区域的匹配问题，因而不具有一般性、实用性。文中提出一种基于二部图最大权匹配的图像相似性度量算法，该算法建立在图像分割的基础上，由于它能有效地解决多区域图像相似性度量问题，并能有效地避免由于分割不准确带来的影响，因此能极大地提高检索的相关性和准确性。     

基于期望与K次方差的信息检索质量评估模型的研究

查全率和查准率是评估信息检索系统检索质量的两个基本标准，长期以来，基于这两个标准，存在着多种评价方法，但是，这些方法基本上是对查全率和查准率做简单的处理，仅反映检索的平均, 对检索稳定性没有分析，并且缺乏一套科学的，系统的评估体系，针对这种情况，借鉴概率学中的期望和方差的思想，用数学语言严格定义了查全期望，查准期望，K次查全方差和K次查准方差等概念，在这些概念的基础上，给出了信息检索质量评估准则，与其它模型相比，该模型能从检索的平均质量和检索的稳定性两方面反映检索系统的性能，因此，对检索质量的评估更加完善和全面。     

基于内容的图像检索系统性能评价

在图像检索需求多元化和专业化的推动下,CBIR技术日趋成熟,目前已有越来越多的商用和科研系统相继推出。这样就迫切需要展开对CBIR系统性能评价标准的研究,因为任何一项技术都是由该领域中相应的评价标准来推动的。为了使人们对这方面的现状动态有一概略了解,首先讨论了基于内容的图像检索评价过程中的两个基本问题,即大规模数据库的建立和获取进行相关性评判,然后对近年来文献中所见的基于内容的图像检索系统性能评价方法进行了回顾和综述;最后在此基础上,对其发展方向进行了探讨,并提出建立一个标准的测试数据集用来推动基于内容的图像检索系统性能评价的发展和更好地将人结合到基于内容的图像检索系统的性能评价过程中,以发展交互式的性能评价方法的建议。     

An effective technique for the content based image retrieval to reduce the semantic gap based on an optimal classifier technique

Content Based Image Retrieval (CBIR) systems use Relevance Feedback (RF) in order to improve the retrieval accuracy. Research focus has been shifted from designing sophisticated low-level feature extraction algorithms to reducing the “semantic gap” between the visual features and the richness of human semantics. In this paper, a novel system is proposed to enhance the gain of long-term relevance feedback. In the proposed system, the general CBIR involves two steps—ABC based training and image retrieval. First, the images other than the query image are pre-processed using median filter and gray scale transformation for removal of noise and resizing. Secondly, the features such as Color, Texture and shape of the image are extracted using Gabor Filter, Gray Level Co-occurrence Matrix and Hu-Moment shape feature techniques and also extract the static features like mean and standard deviation. The extracted features are clustered using k-means algorithm and each cluster are trained using ANN based ABC technique. A method using artificial bee colony (ABC) based artificial neural network (ANN) to update the weights assigned to features by accumulating the knowledge obtained from the user over iterations. Eventually, the comparative analysis performed using the commonly used methods namely precision and recall were clearly shown that the proposed system is suitable for the better CBIR and it can reduce the semantic gap than the conventional systems.     

HBIR: Hypercube-Based Image Retrieval

In this paper, we propose a mapping from low level feature space to the semantic space drawn by the users through relevance feedback to enhance the performance of current content based image retrieval (CBIR) systems. The proposed approach makes a rule base for its inference and configures it using the feedbacks gathered from users during the life cycle of the system. Each rule makes a hypercube (HC) in the feature space corresponding to a semantic concept in the semantic space. Both short and long term strategies are taken to improve the accuracy of the system in response to each feedback of the user and gradually bridge the semantic gap. A scoring paradigm is designed to determine the effective rules and suppress the inefficient ones. For improving the response time, an HC merging approach and, for reducing the conflicts, an HC splitting method is designed. Our experiments on a set of 11000 images from the Corel database show that the proposed approach can better describe the semantic content of images for image retrieval with respect to some existing approaches reported recently in the literature. Moreover, our approach can be better trained and is not saturated in long time, i.e., any feedback improves the precision and recall of the system. Another strength of our method is its ability to address the dynamic nature of the image database such that it can follow the changes occurred instantaneously and permanently by adding and dropping images.     

图像检索中的相似性判别及索引方法综述

基于内容的图像检索(CBIR)是对传统信息检索领域的扩展.它采用图像视觉内容的相似性判别进行查询.CBIR涉及到很多科学领域的课题.本文则仅主要综述CBIR技术中的相似性度量方法,索引方式,以及检索性能的评价.最后,分析了该领域现存的问题、最新研究动态及发展方向.     

An image retrieval system with automatic query modification

Most interactive "query-by-example" based image retrieval systems utilize relevance feedback from the user for bridging the gap between the user's implied concept and the low-level image representation in the database. However, traditional relevance feedback usage in the context of content-based image retrieval (CBIR) may not be very efficient due to a significant overhead in database search and image download time in client-server environments. In this paper, we propose a CBIR system that efficiently addresses the inherent subjectivity in user perception during a retrieval session by employing a novel idea of intra-query modification and learning. The proposed system generates an object-level view of the query image using a new color segmentation technique. Color, shape and spatial features of individual segments are used for image representation and retrieval. The proposed system automatically generates a set of modifications by manipulating the features of the query segment(s). An initial estimate of user perception is learned from the user feedback provided on the set of modified images. This largely improves the precision in the first database search itself and alleviates the overheads of database search and image download. Precision-to-recall ratio is improved in further iterations through a new relevance feedback technique that utilizes both positive as well as negative examples. Extensive experiments have been conducted to demonstrate the feasibility and advantages of the proposed system.     

Meta-metric for saliency detection evaluation metrics based on application preference

Existing saliency detection evaluation metrics often produce inconsistent evaluation results. Because of the widespread application of image saliency detection, we propose a meta-metric to evaluate the performance of these metrics based on the preference of an application that uses saliency maps as weighting maps. This study uses content-based image retrieval (CBIR) as the representative application. First, we perform CBIR using image features extracted from deep convolutional layers of convolutional neural networks as well as saliency maps computed by various saliency detection algorithms as the weighting maps over queries. Second, we establish the preference order of the saliency detection algorithms in the CBIR application by sorting the mean average precision. Third, we determine the preference order of these algorithms using existing saliency detection evaluation metrics. Finally, our meta-metric evaluates these metrics by correlating the preference order in the CBIR application with that determined by each evaluation metric. Experiments on three publicly available datasets show that, of 24 evaluation metrics, the traditional metric: area under receiver operating characteristic curve is the best metric for a CBIR application.     

Experimental analogy of different texture feature extraction techniques in image retrieval systems

Content based image retrieval (CBIR) is an extrusive technique of retrieving the relevant images from vast image archives by extracting their low level features. In this research paper, the pursuance of five most prominent texture feature extraction techniques used in CBIR systems are experimentally compared in detail. The main issue with the CBIR systems is the proper selection of techniques for the extraction of low level features which comprises of color, texture and shape. Among these features, texture is one of the most decisive and dominant features. This selection of features completely depends upon the type of images to be retrieved from the database. The texture techniques explored here are Grey level co-occurrence matrix (GLCM), Discrete wavelet transform (DWT), Gabor transform, Curvelet and Local binary pattern (LBP). These are experimented on three touchstone databases which are Wang, Corel-5 K and Corel-10 K. The chief parameters of CBIR systems are evaluated here such as precision, recall and F-measure on all these databases using all the techniques. After detailed investigation it is figured out that LBP, GLCM and DWT provide highlighted and comparable results in all these datasets in terms of average precision. Besides practical implementation, the précised conceptual examination of these three texture techniques is also proposed in this article. So, this analysis is extremely beneficial for selecting the appropriate feature extraction technique by taking into consideration the experimental results along with image conditions such as noise, rotation etc.

     

A Framework for Benchmarking in CBIR

Content-based image retrieval (CBIR) has been a very active research area for more than ten years. In the last few years the number of publications and retrieval systems produced has become larger and larger. Despite this, there is still no agreed objective way in which to compare the performance of any two of these systems. This fact is blocking the further development of the field since good or promising techniques can not be identified objectively, and the potential commercial success of CBIR systems is hindered because it is hard to establish the quality of an application.We are thus in the position in which other research areas, such as text retrieval or the database systems, found themselves several years ago. To have serious applications, as well as commercial success, objective proof of system quality is needed: in text retrieval the TREC benchmark is a widely accepted performance measure; in the transaction processing field for databases it is the TPC benchmark that has wide support.This paper describes a framework that enables the creation of a benchmark for CBIR. Parts of this framework have already been developed and systems can be evaluated against a small, freely-available database via a web interface. Much work remains to be done with respect to making available large, diverse image databases and obtaining relevance judgments for those large databases. We also need to establish an independent body, accepted by the entire community, that would organize a benchmarking event, give out official results and update the benchmark regularly. The Benchathlon could get this role if it manages to gain the confidence of the field. This should also prevent the negative effects, e.g., benchmarketing, experienced with other benchmarks, such as the TPC predecessors.This paper sets out our ideas for an open framework for performance evaluation. We hope to stimulate discussion on evaluation in image retrieval so that systems can be compared on the same grounds. We also identify query paradigms beyond query by example (QBE) that may be integrated into a benchmarking framework, and we give examples of application-based benchmarking areas.     

Spatial inference and similarity retrieval of an intelligent image database system based on object's spanning representation

In this paper, we presented a novel image representation method to capture the information about spatial relationships between objects in a picture. Our method is more powerful than all other previous methods in terms of accuracy, flexibility, and capability of discriminating pictures. In addition, our method also provides different degrees of granularity for reasoning about directional relations in both 8- and 16-direction reference frames. In similarity retrieval, our system provides twelve types of similarity measures to support flexible matching between the query picture and the database pictures. By exercising a database containing 3600 pictures, we successfully demonstrated the effectiveness of our image retrieval system. Experiment result showed that 97.8% precision rate can be achieved while maintaining 62.5% recall rate; and 97.9% recall rate can be achieved while maintaining 51.7% precision rate. On an average, 86.1% precision rate and 81.2% recall rate can be achieved simultaneously if the threshold is set to 0.5 or 0.6. This performance is considered to be very good as an information retrieval system.     

A new relevance feedback technique for iconic image retrieval based on spatial relationships

Due to the popularity of Internet and the growing demand of image access, the volume of image databases is exploding. Hence, we need a more efficient and effective image searching technology. Relevance feedback technique has been popularly used with content-based image retrieval (CBIR) to improve the precision performance, however, it has never been used with the retrieval systems based on spatial relationships. Hence, we propose a new relevance feedback framework to deal with spatial relationships represented by a specific data structure, called the 2D B_e-string. The notions of relevance estimation and query reformulation are embodied in our method to exploit the relevance knowledge. The irrelevance information is collected in an irrelevant set to rule out undesired pictures and to expedite the convergence speed of relevance feedback. Our system not only handles picture-based relevance feedback, but also deals with region-based feedback mechanism, such that the efficacy and effectiveness of our retrieval system are both satisfactory.     

An interactive approach for CBIR using a network of radial basis functions

An important requirement for constructing effective content-based image retrieval (CBIR) systems is accurate characterization of visual information. Conventional nonadaptive models, which are usually adopted for this task in simple CBIR systems, do not adequately capture all aspects of the characteristics of the human visual system. An effective way of addressing this problem is to adopt a "human-computer" interactive approach, where the users directly teach the system about what they regard as being significant image features and their own notions of image similarity. We propose a machine learning approach for this task, which allows users to directly modify query characteristics by specifying their attributes in the form of training examples. Specifically, we apply a radial-basis function (RBF) network for implementing an adaptive metric which progressively models the notion of image similarity through continual relevance feedback from users. Experimental results show that the proposed methods not only outperform conventional CBIR systems in terms of both accuracy and robustness, but also previously proposed interactive systems.     

Relevance Feedback and Learning in Content-Based Image Search

A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.     

基于内容的图像检索综述

本文简要介绍了基于内容的图像检索，给出了基于内容的图像检索系统的一般结构。对图像检索的发展进行了概述。对基于内容的图像检索的主要研究技术进行了详细和全面的论述，并介绍了几个典型的基于内容的图像检索系统。最后，指出了目前研究中存在的一些主要问题。     

Image retrieval based on regions of interest

Query-by-example is the most popular query model in recent content-based image retrieval (CBIR) systems. A typical query image includes relevant objects (e.g., Eiffel Tower), but also irrelevant image areas (including background). The irrelevant areas limit the effectiveness of existing CBIR systems. To overcome this limitation, the system must be able to determine similarity based on relevant regions alone. We call this class of queries region-of-interest (ROI) queries and propose a technique for processing them in a sampling-based matching framework. A new similarity model is presented and an indexing technique for this new environment is proposed. Our experimental results confirm that traditional approaches, such as Local Color Histogram and Correlogram, suffer from the involvement of irrelevant regions. Our method can handle ROI queries and provide significantly better performance. We also assessed the performance of the proposed indexing technique. The results clearly show that our retrieval procedure is effective for large image data sets.     

Content-based image retrieval using growing hierarchical self-organizing quadtree map

In this paper, a growing hierarchical self-organizing quadtree map (GHSOQM) is proposed and used for a content-based image retrieval (CBIR) system. The incorporation of GHSOQM in a CBIR system organizes images in a hierarchical structure. The retrieval time by GHSOQM is less than that by using direct image comparison using a flat structure. Furthermore, the ability of incremental learning enables GHSOQM to be a prospective neural-network-based approach for CBIR systems. We also propose feature matrices, image distance and relevance feedback for region-based images in the GHSOQM-based CBIR system. Experimental results strongly demonstrate the effectiveness of the proposed system.     

Robust image retrieval with hidden classes

For the purpose of content-based image retrieval (CBIR), image classification is important to help improve the retrieval accuracy and speed of the retrieval process. However, the CBIR systems that employ image classification suffer from the problem of hidden classes. The queries associated with hidden classes cannot be accurately answered using a traditional CBIR system. To address this problem, a robust CBIR scheme is proposed that incorporates a novel query detection technique and a self-adaptive retrieval strategy. A number of experiments carried out on the two popular image datasets demonstrate the effectiveness of the proposed scheme.