Building a user-centered semantic hierarchy in image databases

Recent development in the field of digital media technology has resulted in the generation of a huge number of images. Consequently, content-based image retrieval has emerged as an important area in multimedia computing. Research in human perception of image content suggests that the semantic cues play an important role in image retrieval. In this paper, we present a new paradigm to establish the semantics in image databases based on multi-user relevance feedback. Relevance feedback mechanism is one way to incorporate the users’ perception during image retrieval. By treating each feedback as a weak classifier and combining them together, we are able to capture the categories in the users’ mind and build a user-centered semantic hierarchy in the database to support semantic browsing and searching. We present an image retrieval system based on a city-landscape image database comprising of 3,009 images. We also compare our approach with other typical methods to organize an image database. Superior results have been achieved by the proposed framework.     

图像检索中一种支持多例查询的新方法

基于内容的图像检索的关键问题之一是高层语义和低层图像特征之间的差异,相关反馈技术是缩短这个"语义鸿沟"的有效方法。本文提出了一种新的相关反馈算法,通过分析正例图像在特征空间中的散布来构造该类图像的投影空间,该空间对应于一个语义类在特征空间中分布密集的子空间,在投影空间中计算相似图像。同时根据每次反馈的信息不断修正投影空间来提高系统的检索性能。在Corel大图像库中的实验结果表明,该算法对多例图像查询有较好的检索效果。     

Unsupervised image retrieval framework based on rule base system

This paper introduces unsupervised image retrieval framework based on a rule base system. The proposed framework makes use of geometric moments (GMs) for features extraction. The main advantage with the GMs is that image coordinate transformations can be easily expressed and analyzed in terms of the corresponding transformations in the moment space. These features are used to perform the image mining for acquiring clustering knowledge from a large empirical images database. Irrelevance between images of the same cluster is precisely considered in the proposed framework through a relevant feedback phase followed by a novel clustering refinement model. The images and their corresponding classes pass to a rule base algorithm for extracting a set of accurate rules. These rules are pruning and may reduce the dimensionality of the extracted features. The advantage of the proposed framework is reflected in the retrieval process, which is limited to the images in the class of rule matched with the query image features. Experiments show that the proposed model achieves a very good performance in terms of the average precision, recall and retrieval time compared with other models.     

An efficient semantic – Related image retrieval method

Many previous techniques were designed to retrieve semantic images in a certain neighborhood of the query image and thus bypassing the semantically related images in the whole feature space. Several recently methods were designed to retrieve semantically related images in the entire feature space but with low precision. In this paper, we propose a Semantic – Related Image Retrieval method (SRIR), which can retrieve semantic images spread in the entire feature space with high precision. Our method takes advantage of the user feedback to determine the semantic importance of each query and the importance of each feature. In addition, the retrieval time of our method does not increase with the number of user feedback. We also provide experimental results to demonstrate the effectiveness of our method.     

基于语义学习的图像多模态检索

针对语义鸿沟问题,在语义学习的基础上设计图像的多模态检索系统。该系统结合3种查询方式进行图像检索。基于视觉特征的查询通过特征提取与相似度匹配进行排位。基于标签的查询建立在图像自动标注的基础上,但在语义空间之外的泛化能力较差。基于语义图例的查询能够在很大程度上克服这个缺陷,通过在显式或隐式的语义空间上进行查询,使检索结果更符合人类感知。实验结果表明,与基于纹理特征的图像检索相比,基于语义图例的检索具有更高的精度及召回率。     

集成视觉特征和语义信息的相关反馈方法

为了有效地利用图像检索系统的语义分类信息和视觉特征,提出一种基于Bayes的集成视觉特征和语义信息的相关反馈检索方法.首先,将图像库的数据经语义监督的视觉特征聚类算法划分为小的聚类,每个聚类内数据的视觉特征相似并且语义类别相同;然后以聚类为单位标注正负反馈的实例,这显著区别于以单个图像为单位的相关反馈过程;最后分别以基于视觉特征的Bayes分类器和基于语义的Bayes分类器修正相似距离.在图像库上的实验表明,只用较少的反馈次数就可以达到较高的检索准确率.     

Neighborhood preserving regression for image retrieval

In content-based image retrieval (CBIR), relevance feedback has been proven to be a powerful tool for bridging the gap between low level visual features and high level semantic concepts. Traditionally, relevance feedback driven CBIR is often considered as a supervised learning problem where the user provided feedbacks are used to learn a distance metric or classification function. However, CBIR is intrinsically a semi-supervised learning problem in which the testing samples (images in the database) are present during the learning process. Moreover, when there are no sufficient feedbacks, these methods may suffer from the overfitting problem. In this paper, we propose a novel neighborhood preserving regression algorithm which makes efficient use of both labeled and unlabeled images. By using the unlabeled images, the geometrical structure of the image space can be incorporated into the learning system through a regularizer. Specifically, from all the functions which minimize the empirical loss on the labeled images, we select the one which best preserves the local neighborhood structure of the image space. In this way, our method can obtain a regression function which respects both semantic and geometrical structures of the image database. We present experimental evidence suggesting that our algorithm is able to use unlabeled data effectively for image retrieval.     

Image retrieval using nonlinear manifold embedding

The huge number of images on the Web gives rise to the content-based image retrieval (CBIR) as the text-based search techniques cannot cater to the needs of precisely retrieving Web images. However, CBIR comes with a fundamental flaw: the semantic gap between high-level semantic concepts and low-level visual features. Consequently, relevance feedback is introduced into CBIR to learn the subjective needs of users. However, in practical applications the limited number of user feedbacks is usually overwhelmed by the large number of dimensionalities of the visual feature space. To address this issue, a novel semi-supervised learning method for dimensionality reduction, namely kernel maximum margin projection (KMMP) is proposed in this paper based on our previous work of maximum margin projection (MMP). Unlike traditional dimensionality reduction algorithms such as principal component analysis (PCA) and linear discriminant analysis (LDA), which only see the global Euclidean structure, KMMP is designed for discovering the local manifold structure. After projecting the images into a lower dimensional subspace, KMMP significantly improves the performance of image retrieval. The experimental results on Corel image database demonstrate the effectiveness of our proposed nonlinear algorithm.     

综合底层、语义特征和相关反馈的医学图像检索

医学图像数据库的不断庞大使得医学图像检索成为研究热点。文章根据胸片图像的特点,提出了一种结合图像纹理、形状和语义信息的胸片图像检索方法。同时,还将相关反馈技术融合到算法中。据此,实现了一个图像检索原型系统,依据所设计的评价实验,将不同实验的检索结果进行了比较和分析。实验证明,该文提出的方法具有良好的检索效果。     

图像检索中的动态相似性度量方法

为提高图像检索的效率,近年来相关反馈机制被引入到了基于内容的图像检索领域。该文提出了一种新的相关反馈方法－－动态相似性度量方法。该方法建立在目前被广泛采用的图像相拟性度量方法的基础上,结合了相关反馈图像检索系统的时序特性,通过捕获用户的交互信息,动态地修正图像的相似性度量公式,从而把用户模型嵌入到了图像检索系统,在某种程度上使图像检索结果与人的主观感知更加接近。实验结果表明该方法的性能明显优于其它图像检索系统所采用的方法。     

Automatic content based image retrieval using semantic analysis

We present a new text-to-image re-ranking approach for improving the relevancy rate in searches. In particular, we focus on the fundamental semantic gap that exists between the low-level visual features of the image and high-level textual queries by dynamically maintaining a connected hierarchy in the form of a concept database. For each textual query, we take the results from popular search engines as an initial retrieval, followed by a semantic analysis to map the textual query to higher level concepts. In order to do this, we design a two-layer scoring system which can identify the relationship between the query and the concepts automatically. We then calculate the image feature vectors and compare them with the classifier for each related concept. An image is relevant only when it is related to the query both semantically and content-wise. The second feature of this work is that we loosen the requirement for query accuracy from the user, which makes it possible to perform well on users’ queries containing less relevant information. Thirdly, the concept database can be dynamically maintained to satisfy the variations in user queries, which eliminates the need for human labor in building a sophisticated initial concept database. We designed our experiment using complex queries (based on five scenarios) to demonstrate how our retrieval results are a significant improvement over those obtained from current state-of-the-art image search engines.     

Alternating Feature Spaces in Relevance Feedback

Image retrieval using relevance feedback can be treated as a two-class learning and classification process. The user-labelled relevant and irrelevant images are regarded as positive and negative training samples, based on which a classifier is trained dynamically. Then the classifier in turn classifies all images in the database. In practice, the number of training samples is very small because the users are often impatient. On the other hand, the positive samples usually are not representative since they are the nearest ones to the query and thus less informative. The insufficiency of training samples both in quantities and varieties constrains the generalization ability of the classifier significantly. In this paper, we propose a novel relevance feedback approach, which aims to collect more representative samples and hence improve the performance of classifier. Image labeling and classifier training are conducted in two complementary image feature spaces. Since the samples distribute differently in two spaces, the positive samples may be more informative in one feature space than in another. The two complementary feature spaces are alternated iteratively during the feedback process. To choose appropriate complementary feature spaces, we present two methods to measure the complementarities between two feature spaces quantitatively. Our experimental result on 10,000 images indicates that the proposed feedback approach significantly improves image retrieval performance.     

利用蚁群算法的记忆式图像检索方法

如何有效利用用户的相关反馈信息来进行基于语义的图像检索,是一个具有重要意义并且极具挑战性的问题.介绍了一种基于蚁群算法的记忆式图像检索方法,它是传统记忆式图像检索方法的一种改进.用蚁群算法的思想,利用用户的反馈信息建立图像的语义网络,并依据该语义网络用迭代的方法来检索图像.实验表明,该方法不仅有效,而且存储量小、计算量...     

一种基于近邻保留的相关反馈图像检索算法

图像检索中很多时候会出现相关反馈提供的标注样本数不足,从而导致监督学习方法面临过适应问题的困扰。提出一种能有效使用未标记数据的半监督新型算法:近邻保留回归算法,它通过使已标记数据的观测误差函数最小化,来选择综合性能最好的回归函数,以兼顾图像的语义特征及图像空间的几何结构,并解决过适应问题。实验结果证明,算法能有效提高图像检索系统的性能。     

一种基于稀疏典型性相关分析的图像检索方法

图像语义检索的一个关键问题就是要找到图像底层特征与语义之间的关联,由于文本是表达语义的一种有效手段,因此提出通过研究文本与图像两种模态之间关系来构建反映两者间潜在语义关联的有效模型的思路,基于该模型,可使用自然语言形式(文本语句)来表达检索意图,最终检索到相关图像.该模型基于稀疏典型性相关分析(sparse canonical correlation analysis,简称sparse CCA),按照如下步骤训练得到:首先利用隐语义分析方法构造文本语义空间,然后以视觉词袋(bag of visual words)来表达文本所对应的图像,最后通过Sparse CCA算法找到一个语义相关空间,以实现文本语义与图像视觉单词间的映射.使用稀疏的相关性分析方法可以提高模型可解释性和保证检索结果稳定性.实验结果验证了Sparse CCA方法的有效性,同时也证实了所提出的图像语义检索方法的可行性.     

基于内容的图象检索技术

随着数字图象的日益增多,基于内容的图象检索已成为图象使用者和管理者迫切需要解决的问题,近年来,各国研究者纷纷加入该领域的研究.为了使人们对该领域现状有个概略了解,以推动该领域研究进一步开展,首先概括介绍了基于内容图象检索的产生、发展及其关键技术;然后介绍了特征提取(包括低层特征和语义特征)及其相似性计算、相关反馈等的原理及算法;最后指出了基于内容的图象检索技术与计算机视觉技术的区别所在,并对目前存在的问题和应着重的研究内容以及发展方向进行了分析.     

基于多语义特征融合的图像检索

多媒体技术的发展导致数字图像迅速增长,如何根据语义特征高效检索出满足用户要求的图像,已成为当前各行业迫切需要解决的问题。为此提出一种基于颜色、纹理和形状三种语义特征的图像检索方法,建立了颜色和纹理特征的语义描述,使用BP神经网络实现了低层视觉特征到高层语义特征的映射。选取Corel图像库作为测试图像库,实验通过与基于颜色语义特征的检索方法相比较,取得了良好的实验效果。     

Statistical correlation analysis in image retrieval

A statistical correlation model for image retrieval is proposed. This model captures the semantic relationships among images in a database from simple statistics of user-provided relevance feedback information. It is applied in the post-processing of image retrieval results such that more semantically related images are returned to the user. The algorithm is easy to implement and can be efficiently integrated into an image retrieval system to help improve the retrieval performance. Preliminary experimental results on a database of 100,000 images show that the proposed model could improve image retrieval performance for both content- and text-based queries.     

Active concept learning in image databases.

Concept learning in content-based image retrieval systems is a challenging task. This paper presents an active concept learning approach based on the mixture model to deal with the two basic aspects of a database system: the changing (image insertion or removal) nature of a database and user queries. To achieve concept learning, we a) propose a new user directed semi-supervised expectation-maximization algorithm for mixture parameter estimation, and b) develop a novel model selection method based on Bayesian analysis that evaluates the consistency of hypothesized models with the available information. The analysis of exploitation versus exploration in the search space helps to find the optimal model efficiently. Our concept knowledge transduction approach is able to deal with the cases of image insertion and query images being outside the database. The system handles the situation where users may mislabel images during relevance feedback. Experimental results on Corel database show the efficacy of our active concept learning approach and the improvement in retrieval performance by concept transduction.