Query-by-Gesture: An Alternative Content-Based Image Retrieval Query Scheme

Current research on content-based image retrieval (CBIR) is centered on designing efficient query schemes in order to provide a user with effective mechanisms for image database search. Among representative CBIR query schemes, query-by-sketch has been one of the attractive query tools that are highly adaptive to user's subjectivity. However, query-by-sketch has a few limitations. That is, most sketch tools demand expertise in image processing or computer vision of the user to provide good enough sketches that can be used as query. Furthermore, sketching the exact shape of an object using a mouse can be a burden on the user. To overcome some of the limitations associated with query-by-sketch, we propose a new query method for CBIR, query-by-gesture, that does not require sketches, thereby minimizing user interaction. In our system, the user does not need to use a mouse to make a sketch. Instead, the user draws the shape of the object that heshe intends to search in front of a camera by hand. In addition, our query-by-gesture technique uses relevance feedback to interactively improve retrieval performance and allow progressive refinement of query results according to the user's specification. The efficacy of our proposed method is validated using images from the Corel-Photo CD.     

Relevance Feedback and Learning in Content-Based Image Search

A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.     

基于索引和相关反馈的图像检索研究

在基于图像内容的图像检索(CBIR)系统中,搜索引擎检索图像类似于按照相似标准来查询图像,它应该有足够快的速度并且有较高的检索准确率.索引用来提高系统响应,而相关反馈用于帮助提高检索准确率.在本文中,主要说明基于人感知的相似性度量,以及讨论综合相关反馈的索引方案.该索引方案通过分析特征熵而得出的主从键,而相关反馈是根据Mann-Whitnev检验而提出的,该检验通常用来识别来自同一搜索集中相关图像和不相关图像之间不同特征,并利用不同特征的特点提高检索性能.相关反馈方案针对两不同相似标准来执行,检验判定了这个方法的有效性.最后,把索引机制和相关反馈机制结合起来建立搜索引擎.     

Content-based image retrieval based on a fuzzy approach

A typical content-based image retrieval (CBIR) system would need to handle the vagueness in the user queries as well as the inherent uncertainty in image representation, similarity measure, and relevance feedback. We discuss how fuzzy set theory can be effectively used for this purpose and describe an image retrieval system called FIRST (fuzzy image retrieval system) which incorporates many of these ideas. FIRST can handle exemplar-based, graphical-sketch-based, as well as linguistic queries involving region labels, attributes, and spatial relations. FIRST uses fuzzy attributed relational graphs (FARGs) to represent images, where each node in the graph represents an image region and each edge represents a relation between two regions. The given query is converted to a FARG, and a low-complexity fuzzy graph matching algorithm is used to compare the query graph with the FARGs in the database. The use of an indexing scheme based on a leader clustering algorithm avoids an exhaustive search of the FARG database. We quantify the retrieval performance of the system in terms of several standard measures.     

Content-based image retrieval by using tree-structured features and multi-layer self-organizing map

A new approach for content-based image retrieval (CBIR) is described. In this study, a tree-structured image representation together with a multi-layer self-organizing map (MLSOM) is proposed for efficient image retrieval. In the proposed tree-structured image representation, a root node contains the global features, while child nodes contain the local region-based features. This approach hierarchically integrates more information of image contents to achieve better retrieval accuracy compared with global and region features individually. MLSOM in the proposed method provides effective compression and organization of tree-structured image data. This enables the retrieval system to operate at a much faster rate than that of directly comparing query images with all images in databases. The proposed method also adopts a relevance feedback scheme to improve the retrieval accuracy by a respectable level. Our obtained results indicate that the proposed image retrieval system is robust against different types of image alterations. Comparative results corroborate that the proposed CBIR system is promising in terms of accuracy, speed and robustness.     

Rich Get Richer——图像检索中的一种自适应的相关反馈方法

早期的基于内容的图像检索系统以图像处理技术为核心,研究重点集中在视觉特征的选择和提取方面,而没有充分利用人们在视觉方面的主观性和人类所广泛使用的高层次概念和低层次视觉特征之间的相关性。为解决上述问题,近年来相关反馈在基于内容的图像检索中受到重视。提出了一种新的相关反馈方法,使得高层次语义特征能够逐步嵌入到基于 低层次特征的图像检索中,该方法不仅能够记忆以前的交互信息,而且能够记忆相应的交互信息给系统带来的影响,实验结果表明该方法准确率高、响应速度快。     

Automatic localization and annotation of facial features using machine learning techniques

Content-based image retrieval (CBIR) systems traditionally find images within a database that are similar to query image using low level features, such as colour histograms. However, this requires a user to provide an image to the system. It is easier for a user to query the CBIR system using search terms which requires the image content to be described by semantic labels. However, finding a relationship between the image features and semantic labels is a challenging problem to solve. This paper aims to discover semantic labels for facial features for use in a face image retrieval system. Face image retrieval traditionally uses global face-image information to determine similarity between images. However little has been done in the field of face image retrieval to use local face-features and semantic labelling. Our work aims to develop a clustering method for the discovery of semantic labels of face-features. We also present a machine learning based face-feature localization mechanism which we show has promise in providing accurate localization.     

Content-Based Image Retrieval Based on ROI Detection and Relevance Feedback

Content-based image retrieval is an important research topic in computer vision. We present a new method that combines region of interest (ROI) detection and relevance feedback. The ROI based approach is more accurate in describing the image content than using global features, and the relevance feedback makes the system to be adaptive to subjective human perception. The feedback information is utilized to discover the subjective ROI perception of a particular user, and it is further employed to recompute the features associated with ROIs with the updated personalized ROI preference. A fast computation technique is proposed to avoid repeating the ROI detection for images in the database. It directly estimates the features of the ROIs, which makes the query process fast and efficient. For illustration of the overall approach, we use the color saliency and wavelet feature saliency to determine the ROIs. Normalized projections are selected to represent the shape features associated with the ROIs. Experimental results show that the proposed system has better performance than the global features based approaches and region based techniques without feedback.     

State of the art content based image retrieval techniques using deep learning: a survey

In the recent years the rapid growth of multimedia content makes the image retrieval a challenging research task. Content Based Image Retrieval (CBIR) is a technique which uses features of image to search user required image from large image dataset according to the user’s request in the form of query image. Effective feature representation and similarity measures are very crucial to the retrieval performance of CBIR. The key challenge has been attributed to the well known semantic gap issue. The machine learning has been actively investigated as possible solution to bridge the semantic gap. The recent success of deep learning inspires as a hope for bridging the semantic gap in CBIR. In this paper, we investigate deep learning approach used for CBIR tasks under varied settings from our empirical studies; we find some encouraging conclusions and insights for future research.

     

An effective technique for the content based image retrieval to reduce the semantic gap based on an optimal classifier technique

Content Based Image Retrieval (CBIR) systems use Relevance Feedback (RF) in order to improve the retrieval accuracy. Research focus has been shifted from designing sophisticated low-level feature extraction algorithms to reducing the “semantic gap” between the visual features and the richness of human semantics. In this paper, a novel system is proposed to enhance the gain of long-term relevance feedback. In the proposed system, the general CBIR involves two steps—ABC based training and image retrieval. First, the images other than the query image are pre-processed using median filter and gray scale transformation for removal of noise and resizing. Secondly, the features such as Color, Texture and shape of the image are extracted using Gabor Filter, Gray Level Co-occurrence Matrix and Hu-Moment shape feature techniques and also extract the static features like mean and standard deviation. The extracted features are clustered using k-means algorithm and each cluster are trained using ANN based ABC technique. A method using artificial bee colony (ABC) based artificial neural network (ANN) to update the weights assigned to features by accumulating the knowledge obtained from the user over iterations. Eventually, the comparative analysis performed using the commonly used methods namely precision and recall were clearly shown that the proposed system is suitable for the better CBIR and it can reduce the semantic gap than the conventional systems.     

基于内容的图象检索技术

随着数字图象的日益增多,基于内容的图象检索已成为图象使用者和管理者迫切需要解决的问题,近年来,各国研究者纷纷加入该领域的研究.为了使人们对该领域现状有个概略了解,以推动该领域研究进一步开展,首先概括介绍了基于内容图象检索的产生、发展及其关键技术;然后介绍了特征提取(包括低层特征和语义特征)及其相似性计算、相关反馈等的原理及算法;最后指出了基于内容的图象检索技术与计算机视觉技术的区别所在,并对目前存在的问题和应着重的研究内容以及发展方向进行了分析.     

A unified log-based relevance feedback scheme for image retrieval

Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). In the past, most research efforts in this field have focused on designing effective algorithms for traditional relevance feedback. Given that a CBIR system can collect and store users' relevance feedback information in a history log, an image retrieval system should be able to take advantage of the log data of users' feedback to enhance its retrieval performance. In this paper, we propose a unified framework for log-based relevance feedback that integrates the log of feedback data into the traditional relevance feedback schemes to learn effectively the correlation between low-level image features and high-level concepts. Given the error-prone nature of log data, we present a novel learning technique, named soft label support vector machine, to tackle the noisy data problem. Extensive experiments are designed and conducted to evaluate the proposed algorithms based on the COREL image data set. The promising experimental results validate the effectiveness of our log-based relevance feedback scheme empirically.     

Expert system for color image retrieval

Recently, as Web and various databases contain a large number of images, content-based image retrieval (CBIR) applications are greatly needed. This paper proposes a new image retrieval system using color-spatial information from those applications.First, this paper suggests two kinds of indexing keys to prune away irrelevant images to a given query image: major colors' set (MCS) signature related with color information and distribution block signature (DBS) related with spatial information. After successively applying these filters to a large database, we get only small amount of high potential candidates that are somewhat similar to a query image. Then we make use of the quad modeling (QM) method to set the initial weights of two-dimensional cell in a query image according to each major color. Finally, we retrieve more similar images from the database by comparing a query image with candidate images through a similarity measuring function associated with the weights. In that procedure, we use a new relevance feedback mechanism. This feedback enhances the retrieval effectiveness by dynamically modulating the weights of color-spatial information. Experiments show that the proposed system is not only efficient but also effective.     

A Statistical Framework for Image Category Search from a Mental Picture

Starting from a member of an image database designated the "query image," traditional image retrieval techniques, for example, search by visual similarity, allow one to locate additional instances of a target category residing in the database. However, in many cases, the query image or, more generally, the target category, resides only in the mind of the user as a set of subjective visual patterns, psychological impressions, or "mental pictures." Consequently, since image databases available today are often unstructured and lack reliable semantic annotations, it is often not obvious how to initiate a search session; this is the "page zero problem." We propose a new statistical framework based on relevance feedback to locate an instance of a semantic category in an unstructured image database with no semantic annotation. A search session is initiated from a random sample of images. At each retrieval round, the user is asked to select one image from among a set of displayed images-the one that is closest in his opinion to the target class. The matching is then "mental." Performance is measured by the number of iterations necessary to display an image which satisfies the user, at which point standard techniques can be employed to display other instances. Our core contribution is a Bayesian formulation which scales to large databases. The two key components are a response model which accounts for the user's subjective perception of similarity and a display algorithm which seeks to maximize the flow of information. Experiments with real users and two databases of 20,000 and 60,000 images demonstrate the efficiency of the search process.     

Bayesian relevance feedback for content-based image retrieval

Despite the efforts to reduce the so-called semantic gap between the user's perception of image similarity and the feature-based representation of images, the interaction with the user remains fundamental to improve performances of content-based image retrieval systems. To this end, relevance feedback mechanisms are adopted to refine image-based queries by asking users to mark the set of images retrieved in a neighbourhood of the query as being relevant or not. In this paper, the Bayesian decision theory is used to estimate the boundary between relevant and non-relevant images. Then, a new query is computed whose neighbourhood is likely to fall in a region of the feature space containing relevant images. The performances of the proposed query shifting method have been compared with those of other relevance feedback mechanisms described in the literature. Reported results show the superiority of the proposed method.     

Complementary relevance feedback-based content-based image retrieval

We propose a complementary relevance feedback-based content-based image retrieval (CBIR) system. This system exploits the synergism between short-term and long-term learning techniques to improve the retrieval performance. Specifically, we construct an adaptive semantic repository in long-term learning to store retrieval patterns of historical query sessions. We then extract high-level semantic features from the semantic repository and seamlessly integrate low-level visual features and high-level semantic features in short-term learning to effectively represent the query in a single retrieval session. The high-level semantic features are dynamically updated based on users’ query concept and therefore represent the image’s semantic concept more accurately. Our extensive experimental results demonstrate that the proposed system outperforms its seven state-of-the-art peer systems in terms of retrieval precision and storage space on a large scale imagery database.     

一种基于粗糙集的相关反馈图像检索方法

针对如何在图像检索系统中客观地表达用户的感知，提出了一种基于粗糙集理论的相关反馈算法。通过相关反馈过程将用户感知与图像特征相结合，利用粗糙集理论归纳用户感兴趣的图像语义特征，并根据用户感兴趣的程度调整对应图像特征权重。作者建立了一个实验系统ISS，采用颜色直方图与语义特征作为图像特征，并实现MARS的反馈算法作为性能比较算法。实验结果表明，该算法较MARS系统在检索性能上有较大的提高。     

A new relevance feedback technique for iconic image retrieval based on spatial relationships

Due to the popularity of Internet and the growing demand of image access, the volume of image databases is exploding. Hence, we need a more efficient and effective image searching technology. Relevance feedback technique has been popularly used with content-based image retrieval (CBIR) to improve the precision performance, however, it has never been used with the retrieval systems based on spatial relationships. Hence, we propose a new relevance feedback framework to deal with spatial relationships represented by a specific data structure, called the 2D B_e-string. The notions of relevance estimation and query reformulation are embodied in our method to exploit the relevance knowledge. The irrelevance information is collected in an irrelevant set to rule out undesired pictures and to expedite the convergence speed of relevance feedback. Our system not only handles picture-based relevance feedback, but also deals with region-based feedback mechanism, such that the efficacy and effectiveness of our retrieval system are both satisfactory.