利用蚁群算法的记忆式图像检索方法

如何有效利用用户的相关反馈信息来进行基于语义的图像检索,是一个具有重要意义并且极具挑战性的问题.介绍了一种基于蚁群算法的记忆式图像检索方法,它是传统记忆式图像检索方法的一种改进.用蚁群算法的思想,利用用户的反馈信息建立图像的语义网络,并依据该语义网络用迭代的方法来检索图像.实验表明,该方法不仅有效,而且存储量小、计算量...     

集成视觉特征和语义信息的相关反馈方法

为了有效地利用图像检索系统的语义分类信息和视觉特征,提出一种基于Bayes的集成视觉特征和语义信息的相关反馈检索方法.首先,将图像库的数据经语义监督的视觉特征聚类算法划分为小的聚类,每个聚类内数据的视觉特征相似并且语义类别相同;然后以聚类为单位标注正负反馈的实例,这显著区别于以单个图像为单位的相关反馈过程;最后分别以基于视觉特征的Bayes分类器和基于语义的Bayes分类器修正相似距离.在图像库上的实验表明,只用较少的反馈次数就可以达到较高的检索准确率.     

Instance based personalized multi-form image browsing and retrieval

It is important to adapt and personalize image browsing and retrieval systems based on users’ preferences for improved user experience and satisfaction. In this paper, we present a novel instance based personalized multi-form image representation with implicit relevance feedback and adaptive weighting approach for image browsing and retrieval systems. In the proposed system, images are grouped into forms, which represent different information on images such as location, content etc. We conducted user interviews on image browsing, sharing and retrieval systems for understanding image browsing and searching behaviors of users. Based on the insights gained from the user interview study we propose an adaptive weighting method and implicit relevance feedback for multi-form structures that aim to improve the efficiency and accuracy of the system. Statistics of the past actions are considered for modeling the target of the users. Thus, on each iteration weights of the forms are updated adaptively. Moreover, retrieval results are modified according to the users’ preferences on iterations in order to improve personalized user experience. The proposed method has been evaluated and results are illustrated in the paper. It is shown that, satisfactory improvements can be achieved with proposed approaches in the multi-form scheme.     

基于内容的图象检索技术

随着数字图象的日益增多,基于内容的图象检索已成为图象使用者和管理者迫切需要解决的问题,近年来,各国研究者纷纷加入该领域的研究.为了使人们对该领域现状有个概略了解,以推动该领域研究进一步开展,首先概括介绍了基于内容图象检索的产生、发展及其关键技术;然后介绍了特征提取(包括低层特征和语义特征)及其相似性计算、相关反馈等的原理及算法;最后指出了基于内容的图象检索技术与计算机视觉技术的区别所在,并对目前存在的问题和应着重的研究内容以及发展方向进行了分析.     

A Statistical Framework for Image Category Search from a Mental Picture

Starting from a member of an image database designated the "query image," traditional image retrieval techniques, for example, search by visual similarity, allow one to locate additional instances of a target category residing in the database. However, in many cases, the query image or, more generally, the target category, resides only in the mind of the user as a set of subjective visual patterns, psychological impressions, or "mental pictures." Consequently, since image databases available today are often unstructured and lack reliable semantic annotations, it is often not obvious how to initiate a search session; this is the "page zero problem." We propose a new statistical framework based on relevance feedback to locate an instance of a semantic category in an unstructured image database with no semantic annotation. A search session is initiated from a random sample of images. At each retrieval round, the user is asked to select one image from among a set of displayed images-the one that is closest in his opinion to the target class. The matching is then "mental." Performance is measured by the number of iterations necessary to display an image which satisfies the user, at which point standard techniques can be employed to display other instances. Our core contribution is a Bayesian formulation which scales to large databases. The two key components are a response model which accounts for the user's subjective perception of similarity and a display algorithm which seeks to maximize the flow of information. Experiments with real users and two databases of 20,000 and 60,000 images demonstrate the efficiency of the search process.     

HBIR: Hypercube-Based Image Retrieval

In this paper, we propose a mapping from low level feature space to the semantic space drawn by the users through relevance feedback to enhance the performance of current content based image retrieval (CBIR) systems. The proposed approach makes a rule base for its inference and configures it using the feedbacks gathered from users during the life cycle of the system. Each rule makes a hypercube (HC) in the feature space corresponding to a semantic concept in the semantic space. Both short and long term strategies are taken to improve the accuracy of the system in response to each feedback of the user and gradually bridge the semantic gap. A scoring paradigm is designed to determine the effective rules and suppress the inefficient ones. For improving the response time, an HC merging approach and, for reducing the conflicts, an HC splitting method is designed. Our experiments on a set of 11000 images from the Corel database show that the proposed approach can better describe the semantic content of images for image retrieval with respect to some existing approaches reported recently in the literature. Moreover, our approach can be better trained and is not saturated in long time, i.e., any feedback improves the precision and recall of the system. Another strength of our method is its ability to address the dynamic nature of the image database such that it can follow the changes occurred instantaneously and permanently by adding and dropping images.     

Complementary relevance feedback-based content-based image retrieval

We propose a complementary relevance feedback-based content-based image retrieval (CBIR) system. This system exploits the synergism between short-term and long-term learning techniques to improve the retrieval performance. Specifically, we construct an adaptive semantic repository in long-term learning to store retrieval patterns of historical query sessions. We then extract high-level semantic features from the semantic repository and seamlessly integrate low-level visual features and high-level semantic features in short-term learning to effectively represent the query in a single retrieval session. The high-level semantic features are dynamically updated based on users’ query concept and therefore represent the image’s semantic concept more accurately. Our extensive experimental results demonstrate that the proposed system outperforms its seven state-of-the-art peer systems in terms of retrieval precision and storage space on a large scale imagery database.     

基于多语义特征融合的图像检索

多媒体技术的发展导致数字图像迅速增长,如何根据语义特征高效检索出满足用户要求的图像,已成为当前各行业迫切需要解决的问题。为此提出一种基于颜色、纹理和形状三种语义特征的图像检索方法,建立了颜色和纹理特征的语义描述,使用BP神经网络实现了低层视觉特征到高层语义特征的映射。选取Corel图像库作为测试图像库,实验通过与基于颜色语义特征的检索方法相比较,取得了良好的实验效果。     

Automatic localization and annotation of facial features using machine learning techniques

Content-based image retrieval (CBIR) systems traditionally find images within a database that are similar to query image using low level features, such as colour histograms. However, this requires a user to provide an image to the system. It is easier for a user to query the CBIR system using search terms which requires the image content to be described by semantic labels. However, finding a relationship between the image features and semantic labels is a challenging problem to solve. This paper aims to discover semantic labels for facial features for use in a face image retrieval system. Face image retrieval traditionally uses global face-image information to determine similarity between images. However little has been done in the field of face image retrieval to use local face-features and semantic labelling. Our work aims to develop a clustering method for the discovery of semantic labels of face-features. We also present a machine learning based face-feature localization mechanism which we show has promise in providing accurate localization.     

Content-Based Image Retrieval Based on ROI Detection and Relevance Feedback

Content-based image retrieval is an important research topic in computer vision. We present a new method that combines region of interest (ROI) detection and relevance feedback. The ROI based approach is more accurate in describing the image content than using global features, and the relevance feedback makes the system to be adaptive to subjective human perception. The feedback information is utilized to discover the subjective ROI perception of a particular user, and it is further employed to recompute the features associated with ROIs with the updated personalized ROI preference. A fast computation technique is proposed to avoid repeating the ROI detection for images in the database. It directly estimates the features of the ROIs, which makes the query process fast and efficient. For illustration of the overall approach, we use the color saliency and wavelet feature saliency to determine the ROIs. Normalized projections are selected to represent the shape features associated with the ROIs. Experimental results show that the proposed system has better performance than the global features based approaches and region based techniques without feedback.     

Neighborhood preserving regression for image retrieval

In content-based image retrieval (CBIR), relevance feedback has been proven to be a powerful tool for bridging the gap between low level visual features and high level semantic concepts. Traditionally, relevance feedback driven CBIR is often considered as a supervised learning problem where the user provided feedbacks are used to learn a distance metric or classification function. However, CBIR is intrinsically a semi-supervised learning problem in which the testing samples (images in the database) are present during the learning process. Moreover, when there are no sufficient feedbacks, these methods may suffer from the overfitting problem. In this paper, we propose a novel neighborhood preserving regression algorithm which makes efficient use of both labeled and unlabeled images. By using the unlabeled images, the geometrical structure of the image space can be incorporated into the learning system through a regularizer. Specifically, from all the functions which minimize the empirical loss on the labeled images, we select the one which best preserves the local neighborhood structure of the image space. In this way, our method can obtain a regression function which respects both semantic and geometrical structures of the image database. We present experimental evidence suggesting that our algorithm is able to use unlabeled data effectively for image retrieval.     

一种基于SVM的相关反馈图像检索算法

相关反馈技术是近年来图像检索中的研究热点,本文以MPEG-7的边缘直方图作为图像特征,以支持向量机（SVM）为分类器,提出一种新的相关反馈算法。在每次反馈中对用户标记的相关样本进行学习,用历次返回的结果更新训练样本集,建立SVM分类器模型,并根据模型进行检索。本文还对不同核函数的SVM进行了对比,得出RBF核函数的SVM有较高的检索精度。使用由10000幅图像组成的图像库进行实验,结果表明,该算法可有效地检索出更多的相关图像,并且在有限训练样本情况下具有良好的泛化能力。     

图像检索中的动态相似性度量方法

为提高图像检索的效率,近年来相关反馈机制被引入到了基于内容的图像检索领域。该文提出了一种新的相关反馈方法－－动态相似性度量方法。该方法建立在目前被广泛采用的图像相拟性度量方法的基础上,结合了相关反馈图像检索系统的时序特性,通过捕获用户的交互信息,动态地修正图像的相似性度量公式,从而把用户模型嵌入到了图像检索系统,在某种程度上使图像检索结果与人的主观感知更加接近。实验结果表明该方法的性能明显优于其它图像检索系统所采用的方法。     

图像的语义关联与图像索引

基于内容图像检索的主要挑战在于不断变化的图像检索要求、难于表达的图像内容以及图像表达的数字阵列与通常可以被人类所接受的概念化内容之间的语义鸿沟.提出了一个基于语义关联的图像检索方法,在语义关联的基础上形成一个场景类别的语义表达,以便用户可以将感知上相似的图像组织在一起,形成概念上下文,使得用户可以解释和标记图像而无需给...     

Image Collection Organization and Its Application to Indexing, Browsing, Summarization, and Semantic Retrieval

In this paper, we present a new framework for organizing image collections into structures that can be used for indexing, browsing, retrieval and summarization. Instead of using tree-based techniques which are not suitable for images, we develop a new solution that is specifically designed for image collections. We consider both low-level image content and high-level semantics in an attempt to alleviate the semantic gap encountered by many systems. The fact that our model is based on a probabilistic framework makes it possible to combine it in a natural way with probabilistic techniques developed recently for image retrieval. The structure our model generates is applied for four purposes. The first is to provide retrieval module with an index, which allows it to improve retrieval time and accuracy, while the second is to provide users with a hierarchical browsing catalog that allows them to navigate the image collection by subject. This represents an additional step towards facilitating human-computer interaction in the context of image retrieval and navigation. The third aim is to provide users with a summarization of the general content of each class in the collection, and the fourth is a retrieval mechanism. Related issues such as relevance feedback and feature selection are also addressed. The experiments at the end of the paper show that the proposed framework yields some significant improvements     

An image retrieval system with automatic query modification

Most interactive "query-by-example" based image retrieval systems utilize relevance feedback from the user for bridging the gap between the user's implied concept and the low-level image representation in the database. However, traditional relevance feedback usage in the context of content-based image retrieval (CBIR) may not be very efficient due to a significant overhead in database search and image download time in client-server environments. In this paper, we propose a CBIR system that efficiently addresses the inherent subjectivity in user perception during a retrieval session by employing a novel idea of intra-query modification and learning. The proposed system generates an object-level view of the query image using a new color segmentation technique. Color, shape and spatial features of individual segments are used for image representation and retrieval. The proposed system automatically generates a set of modifications by manipulating the features of the query segment(s). An initial estimate of user perception is learned from the user feedback provided on the set of modified images. This largely improves the precision in the first database search itself and alleviates the overheads of database search and image download. Precision-to-recall ratio is improved in further iterations through a new relevance feedback technique that utilizes both positive as well as negative examples. Extensive experiments have been conducted to demonstrate the feasibility and advantages of the proposed system.     

一种基于记忆的图像检索相关反馈策略

相关反馈在基于内容的图像检索中成为提升检索效率的一项重要技术。然而,在图像检索中,高层语义与底层特征之间存在着巨大的"语义鸿沟",传统的相关反馈技术需要多次反馈才能获得满意结果,这使得用户的检索任务既耗时且繁琐。因此,本文通过对图像检索反馈日志信息的存储及使用过程进行分析,提出一种新的基于记忆的相关反馈策略。通过原型系统实验,与传统反馈策略相比,本文提出的策略对检索效率有明显改善。     

Relevance Feedback and Learning in Content-Based Image Search

A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.     

A New Approach to Object-Related Image Retrieval

Image retrieval has been commonly attempted using non-semantic approaches. It is clear though, that semantic retrieval is more desirable because it facilitates the user's task. In this paper, we present a new approach to semantic access of a database of images by asking for the presence of certain objects; this is known as object-related image retrieval.This approach is built within a classical computer vision framework (i.e. localization, segmentation and identification). Our approach first searches for the main areas of attention (most salient areas of an image) and then applies appearance-based methods to classify (index) all images by ‘symbolic’ names. These names are referred to objects, which finally allows the use of semantics driven by these object names, e.g. retrieve ‘all those images that have a bull and Melissa's face'.The use of a totally automatic system would cause some errors of indexing (and so retrieval). To solve this we use a human-in-the-loop strategy where a human expert is placed after the two outputs of the system to confirm their ‘correctness’. An experimental result using a database of 3000 images is presented.     

Automatic content based image retrieval using semantic analysis

We present a new text-to-image re-ranking approach for improving the relevancy rate in searches. In particular, we focus on the fundamental semantic gap that exists between the low-level visual features of the image and high-level textual queries by dynamically maintaining a connected hierarchy in the form of a concept database. For each textual query, we take the results from popular search engines as an initial retrieval, followed by a semantic analysis to map the textual query to higher level concepts. In order to do this, we design a two-layer scoring system which can identify the relationship between the query and the concepts automatically. We then calculate the image feature vectors and compare them with the classifier for each related concept. An image is relevant only when it is related to the query both semantically and content-wise. The second feature of this work is that we loosen the requirement for query accuracy from the user, which makes it possible to perform well on users’ queries containing less relevant information. Thirdly, the concept database can be dynamically maintained to satisfy the variations in user queries, which eliminates the need for human labor in building a sophisticated initial concept database. We designed our experiment using complex queries (based on five scenarios) to demonstrate how our retrieval results are a significant improvement over those obtained from current state-of-the-art image search engines.