基于内容的交互式感性图象检索

随着信息化社会的到来及信息高速公路计划的实施,人们越来越多地接触到大量的图象信息,因此基于内容的图象检索已经成为当前的一个热门研究课题,并在多媒体数据库、电子图书馆、商标管理、医疗图象管理、公安系统、卫星图象管理等方面得到广泛应用。然而,大多数基于内容的图象检索系统主要是通过图象多维物理特征的相似性匹配来进行查询,而对于用户的爱好、情感等主观或感性化的因素则考虑较少。为了弥补这方面的不足,提出了一种基于内容的交互式感性图象检索方法。该方法采用交互式进化算法,并通过人机交互的方式,来将用户的直觉、情感等感性化的因素融入到进化过程,以便进行图象的交互式在线检索;针对在检索过程中,因进化的时间可能较长和因需要用户确定的适应度值较多而产生的用户疲劳问题,采用神经网络离线学习的方法来减轻用户疲劳,从而实现了根据用户的情感和基于图象内容的图象检索,并取得了较好的实验结果。     

Content-based image retrieval using fuzzy perceptual feedback

In this paper, a new framework called fuzzy relevance feedback in interactive content-based image retrieval (CBIR) systems is introduced. Conventional binary labeling scheme in relevance feedback requires a crisp decision to be made on the relevance of the retrieved images. However, it is inflexible as user interpretation of visual content varies with respect to different information needs and perceptual subjectivity. In addition, users tend to learn from the retrieval results to further refine their information requests. It is, therefore, inadequate to describe the user’s fuzzy perception of image similarity with crisp logic. In view of this, we propose a fuzzy relevance feedback approach which enables the user to make a fuzzy judgement. It integrates the user’s fuzzy interpretation of visual content into the notion of relevance feedback. An efficient learning approach is proposed using a fuzzy radial basis function (FRBF) network. The network is constructed based on the user’s feedbacks. The underlying network parameters are optimized by adopting a gradient-descent training strategy due to its computational efficiency. Experimental results using a database of 10,000 images demonstrate the effectiveness of the proposed method.

图像检索中的动态相似性度量方法

为提高图像检索的效率,近年来相关反馈机制被引入到了基于内容的图像检索领域。该文提出了一种新的相关反馈方法－－动态相似性度量方法。该方法建立在目前被广泛采用的图像相拟性度量方法的基础上,结合了相关反馈图像检索系统的时序特性,通过捕获用户的交互信息,动态地修正图像的相似性度量公式,从而把用户模型嵌入到了图像检索系统,在某种程度上使图像检索结果与人的主观感知更加接近。实验结果表明该方法的性能明显优于其它图像检索系统所采用的方法。     

A survey of browsing models for content based image retrieval

The problem of content based image retrieval (CBIR) has traditionally been investigated within a framework that emphasises the explicit formulation of a query: users initiate an automated search for relevant images by submitting an image or draw a sketch that exemplifies their information need. Often, relevance feedback is incorporated as a post-retrieval step for optimising the way evidence from different visual features is combined. While this sustained methodological focus has helped CBIR to mature, it has also brought out its limitations more clearly: There is often little support for exploratory search and scaling to very large collections is problematic. Moreover, the assumption that users are always able to formulate an appropriate query is questionable. An effective, albeit much less studied, method of accessing image collections based on visual content is that of browsing. The aim of this survey paper is to provide a structured overview of the different models that have been explored over the last one to two decades, to highlight the particular challenges of the browsing approach and to focus attention on a few interesting issues that warrant more intense research.

Similarity learning for texture image retrieval using multiple classifier system

Multiple Classifier System has found its applications in many areas such as handwriting recognition, speaker recognition, medical diagnosis, fingerprint recognition, personal identification and others. However, there have been rare attempts to develop content-based image retrieval (CBIR) system that uses multiple classifiers to learn visual similarity. Texture as a primitive visual content is often used in many important applications (viz. Medical image analysis and medical CBIR system). In this paper, a texture image retrieval system is developed that learns the visual similarity in terms of class membership using multiple classifiers. The way proposed approach combines the decisions of multiple classifiers to obtain final class memberships of query for each of the output classes is also a novel concept. A modified distance that is weighted with the membership values obtained through similarity learning is used for ranking. Three different algorithms are proposed for the retrieval of images against a query image displaying the strength of multiple classifier approach, class membership score and their interplay to achieve the objective defined in terms of simplicity, retrieval effectiveness and speed. The proposed methods based on multiple classifiers achieve higher retrieval accuracy with lower standard deviation compared to all the competing methods irrespective of the texture database and feature set used. The multiple classifier retrieval schemes proposed here is tested for texture image retrieval. However, these can be used for any other challenging retrieval problems.     

Collaborative image retrieval via regularized metric learning

In content-based image retrieval (CBIR), relevant images are identified based on their similarities to query images. Most CBIR algorithms are hindered by the semantic gap between the low-level image features used for computing image similarity and the high-level semantic concepts conveyed in images. One way to reduce the semantic gap is to utilize the log data of users' feedback that has been collected by CBIR systems in history, which is also called “collaborative image retrieval.” In this paper, we present a novel metric learning approach, named “regularized metric learning,” for collaborative image retrieval, which learns a distance metric by exploring the correlation between low-level image features and the log data of users' relevance judgments. Compared to the previous research, a regularization mechanism is used in our algorithm to effectively prevent overfitting. Meanwhile, we formulate the proposed learning algorithm into a semidefinite programming problem, which can be solved very efficiently by existing software packages and is scalable to the size of log data. An extensive set of experiments has been conducted to show that the new algorithm can substantially improve the retrieval accuracy of a baseline CBIR system using Euclidean distance metric, even with a modest amount of log data. The experiment also indicates that the new algorithm is more effective and more efficient than two alternative algorithms, which exploit log data for image retrieval.     

An Efficient Deep Learning-based Content-based Image Retrieval Framework

The use of massive image databases has increased drastically over the few years due to evolution of multimedia technology. Image retrieval has become one of the vital tools in image processing applications. Content-Based Image Retrieval (CBIR) has been widely used in varied applications. But, the results produced by the usage of a single image feature are not satisfactory. So, multiple image features are used very often for attaining better results. But, fast and effective searching for relevant images from a database becomes a challenging task. In the previous existing system, the CBIR has used the combined feature extraction technique using color auto-correlogram, Rotation-Invariant Uniform Local Binary Patterns (RULBP) and local energy. However, the existing system does not provide significant results in terms of recall and precision. Also, the computational complexity is higher for the existing CBIR systems. In order to handle the above mentioned issues, the Gray Level Co-occurrence Matrix (GLCM) with Deep Learning based Enhanced Convolution Neural Network (DLECNN) is proposed in this work. The proposed system framework includes noise reduction using histogram equalization, feature extraction using GLCM, similarity matching computation using Hierarchal and Fuzzy c- Means (HFCM) algorithm and the image retrieval using DLECNN algorithm. The histogram equalization has been used for computing the image enhancement. This enhanced image has a uniform histogram. Then, the GLCM method has been used to extract the features such as shape, texture, colour, annotations and keywords. The HFCM similarity measure is used for computing the query image vector's similarity index with every database images. For enhancing the performance of this image retrieval approach, the DLECNN algorithm is proposed to retrieve more accurate features of the image. The proposed GLCM+DLECNN algorithm provides better results associated with high accuracy, precision, recall, f-measure and lesser complexity. From the experimental results, it is clearly observed that the proposed system provides efficient image retrieval for the given query image.     

Towards Data-Adaptive and User-Adaptive Image Retrieval by Peer Indexing

Adaptation to the characteristics of specific images and the preferences of individual users is critical to the success of an image retrieval system but insufficiently addressed by the existing approaches. In this paper, we propose an elegant and effective approach to data-adaptive and user-adaptive image retrieval based on the idea of peer indexing—describing an image through semantically relevant peer images. Specifically, we associate each image with a two-level peer index that models the “data characteristics” of the image as well as the “user characteristics” of individual users with respect to this image. Based on two-level image peer indexes, a set of retrieval parameters including query vectors and similarity metric are optimized towards both data and user characteristics by applying the pseudo feedback strategy. A cooperative framework is proposed under which peer indexes and image visual features are integrated to facilitate data- and user-adaptive image retrieval. Simulation experiments conducted on real-world images have verified the effectiveness of our approach in a relatively restricted setting.     

Relevance Feedback and Learning in Content-Based Image Search

A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.     

基于内容的图象检索技术

随着数字图象的日益增多,基于内容的图象检索已成为图象使用者和管理者迫切需要解决的问题,近年来,各国研究者纷纷加入该领域的研究.为了使人们对该领域现状有个概略了解,以推动该领域研究进一步开展,首先概括介绍了基于内容图象检索的产生、发展及其关键技术;然后介绍了特征提取(包括低层特征和语义特征)及其相似性计算、相关反馈等的原理及算法;最后指出了基于内容的图象检索技术与计算机视觉技术的区别所在,并对目前存在的问题和应着重的研究内容以及发展方向进行了分析.     

Combining global and local matching of multiple features for precise item image retrieval

With the fast-growing of online shopping services, there are millions even billions of commercial item images available on the Internet. How to effectively leverage visual search method to find the items of users’ interests is an important yet challenging task. Besides global appearances (e.g., color, shape or pattern), users may often pay more attention to the local styles of certain products, thus an ideal visual item search engine should support detailed and precise search of similar images, which is beyond the capabilities of current search systems. In this paper, we propose a novel system named iSearch and global/local matching of local features are combined to do precise retrieval of item images in an interactive manner. We extract multiple local features including scale-invariant feature transform (SIFT), regional color moments and object contour fragments to sufficiently represent the visual appearances of items; while global and local matching of large-scale image dataset are allowed. To do this, an effective contour fragments encoding and indexing method is developed. Meanwhile, to improve the matching robustness of local features, we encode the spatial context with grid representations and a simple but effective verification approach using triangle relations constraints is proposed for spatial consistency filtering. The experimental evaluations show the promising results of our approach and system.     

Image retrieval based on regions of interest

Query-by-example is the most popular query model in recent content-based image retrieval (CBIR) systems. A typical query image includes relevant objects (e.g., Eiffel Tower), but also irrelevant image areas (including background). The irrelevant areas limit the effectiveness of existing CBIR systems. To overcome this limitation, the system must be able to determine similarity based on relevant regions alone. We call this class of queries region-of-interest (ROI) queries and propose a technique for processing them in a sampling-based matching framework. A new similarity model is presented and an indexing technique for this new environment is proposed. Our experimental results confirm that traditional approaches, such as Local Color Histogram and Correlogram, suffer from the involvement of irrelevant regions. Our method can handle ROI queries and provide significantly better performance. We also assessed the performance of the proposed indexing technique. The results clearly show that our retrieval procedure is effective for large image data sets.     

An image retrieval system with automatic query modification

Most interactive "query-by-example" based image retrieval systems utilize relevance feedback from the user for bridging the gap between the user's implied concept and the low-level image representation in the database. However, traditional relevance feedback usage in the context of content-based image retrieval (CBIR) may not be very efficient due to a significant overhead in database search and image download time in client-server environments. In this paper, we propose a CBIR system that efficiently addresses the inherent subjectivity in user perception during a retrieval session by employing a novel idea of intra-query modification and learning. The proposed system generates an object-level view of the query image using a new color segmentation technique. Color, shape and spatial features of individual segments are used for image representation and retrieval. The proposed system automatically generates a set of modifications by manipulating the features of the query segment(s). An initial estimate of user perception is learned from the user feedback provided on the set of modified images. This largely improves the precision in the first database search itself and alleviates the overheads of database search and image download. Precision-to-recall ratio is improved in further iterations through a new relevance feedback technique that utilizes both positive as well as negative examples. Extensive experiments have been conducted to demonstrate the feasibility and advantages of the proposed system.     

基于内容图像检索中的特征性能评价

在基于内容的图像检索中，不同图像特征反映了图像各个侧面的内在特性，因此，在使用图像特征进行检索时存在多种相似性度量方法．特征以及特征间相似性度量方法的选取是当前CBIR研究的一个重要课题．评估了CBIR系统中使用的图像特征在不同相似性度量方法下及多种特征在不同图像库上的检索性能，为CBIR系统的设计和实现提供一定的依据．通过实验发现，图像特征的检索性能不仅同相似性度量方法有关系，同时与图像库也有密切的关系．     

综合颜色和纹理特征的图像检索算法

基于内容的图像检索是当前多媒体信息检索的热点之一。基于内容的图像检索技术是根据对图像内容(特征)的描述和提取,在图像库中找到具有指定内容(特征)的图像。本文对图像颜色特征和纹理特征的提取、相似性度量等基于内容的图像检索的关键技术进行了分析和研究,并在此基础上,提出了一个基于颜色特征和纹理特征的图像检索算法并验证了其有效性。该算法采用HSV颜色空间的直方图作为颜色特征向量,采用灰度共生矩阵的四个纹理特征:能量、熵、惯性矩和相关性构成纹理特征向量,采用欧氏距离进行相似性度量。实验结果表明,该算法实现的系统具有良好的图像检索功能。     

Automatic machine interactions for content-based image retrieval using a self-organizing tree map architecture

In this paper, an unsupervised learning network is explored to incorporate a self-learning capability into image retrieval systems. Our proposal is a new attempt to automate recursive content-based image retrieval. The adoption of a self-organizing tree map (SOTM) is introduced, to minimize the user participation in an effort to automate interactive retrieval. The automatic learning mode has been applied to optimize the relevance feedback (RF) method and the single radial basis function-based RF method. In addition, a semiautomatic version is proposed to support retrieval with different user subjectivities. Image similarity is evaluated by a nonlinear model, which performs discrimination based on local analysis. Experimental results show robust and accurate performance by the proposed method, as compared with conventional noninteractive content-based image retrieval (CBIR) systems and user controlled interactive systems, when applied to image retrieval in compressed and uncompressed image databases.