Instance based personalized multi-form image browsing and retrieval

It is important to adapt and personalize image browsing and retrieval systems based on users’ preferences for improved user experience and satisfaction. In this paper, we present a novel instance based personalized multi-form image representation with implicit relevance feedback and adaptive weighting approach for image browsing and retrieval systems. In the proposed system, images are grouped into forms, which represent different information on images such as location, content etc. We conducted user interviews on image browsing, sharing and retrieval systems for understanding image browsing and searching behaviors of users. Based on the insights gained from the user interview study we propose an adaptive weighting method and implicit relevance feedback for multi-form structures that aim to improve the efficiency and accuracy of the system. Statistics of the past actions are considered for modeling the target of the users. Thus, on each iteration weights of the forms are updated adaptively. Moreover, retrieval results are modified according to the users’ preferences on iterations in order to improve personalized user experience. The proposed method has been evaluated and results are illustrated in the paper. It is shown that, satisfactory improvements can be achieved with proposed approaches in the multi-form scheme.     

Building a user-centered semantic hierarchy in image databases

Recent development in the field of digital media technology has resulted in the generation of a huge number of images. Consequently, content-based image retrieval has emerged as an important area in multimedia computing. Research in human perception of image content suggests that the semantic cues play an important role in image retrieval. In this paper, we present a new paradigm to establish the semantics in image databases based on multi-user relevance feedback. Relevance feedback mechanism is one way to incorporate the users’ perception during image retrieval. By treating each feedback as a weak classifier and combining them together, we are able to capture the categories in the users’ mind and build a user-centered semantic hierarchy in the database to support semantic browsing and searching. We present an image retrieval system based on a city-landscape image database comprising of 3,009 images. We also compare our approach with other typical methods to organize an image database. Superior results have been achieved by the proposed framework.     

一种基于粗糙集的相关反馈图像检索方法

针对如何在图像检索系统中客观地表达用户的感知，提出了一种基于粗糙集理论的相关反馈算法。通过相关反馈过程将用户感知与图像特征相结合，利用粗糙集理论归纳用户感兴趣的图像语义特征，并根据用户感兴趣的程度调整对应图像特征权重。作者建立了一个实验系统ISS，采用颜色直方图与语义特征作为图像特征，并实现MARS的反馈算法作为性能比较算法。实验结果表明，该算法较MARS系统在检索性能上有较大的提高。     

A new relevance feedback technique for iconic image retrieval based on spatial relationships

Due to the popularity of Internet and the growing demand of image access, the volume of image databases is exploding. Hence, we need a more efficient and effective image searching technology. Relevance feedback technique has been popularly used with content-based image retrieval (CBIR) to improve the precision performance, however, it has never been used with the retrieval systems based on spatial relationships. Hence, we propose a new relevance feedback framework to deal with spatial relationships represented by a specific data structure, called the 2D B_e-string. The notions of relevance estimation and query reformulation are embodied in our method to exploit the relevance knowledge. The irrelevance information is collected in an irrelevant set to rule out undesired pictures and to expedite the convergence speed of relevance feedback. Our system not only handles picture-based relevance feedback, but also deals with region-based feedback mechanism, such that the efficacy and effectiveness of our retrieval system are both satisfactory.     

基于模糊区域的CT脑图像检索及关联反馈

基于模糊区域特征的图像检索算法和关联反馈算法是当前图像检索领域的研究热点,由于区域模糊相似度的复杂性,绝大多数关联反馈算法不能应用到基于模糊区域特征的图像检索方法中。为解决这个问题,论文修改了模糊相似度计算方法,并结合经典的基于权重调整的关联反馈算法,提出一种基于模糊区域特征的关联反馈算法。对脑出血CT图像的检索实验结果表明该算法效果较好。     

基于强化学习的相关反馈图像检索算法

相关反馈算法是图像检索不可缺的重要组成部分,是近来图像检索中研究的一个热点。提出了基于强化学习的相关反馈算法。根据强化学习中的Q_学习函数,建立矩阵Q,对每幅图像建立对应的一项Q_i（i=1,2,…,n）,记录每幅图像的本次检索中的累计反馈值,并根据加权特征法计算新的特征,对于每幅反馈的图像根据Q_学习函数计算其当前的累计反馈值。Q值越大即越与例子图像相关。由于强化学习是通过不断对环境的反馈来获得最佳的路径,这与相关反馈通过对用户检索意图的摸索来获得最优答案的思想一致。实验表明,提出的相关反馈算法具有更大的优越性。     

Complementary relevance feedback-based content-based image retrieval

We propose a complementary relevance feedback-based content-based image retrieval (CBIR) system. This system exploits the synergism between short-term and long-term learning techniques to improve the retrieval performance. Specifically, we construct an adaptive semantic repository in long-term learning to store retrieval patterns of historical query sessions. We then extract high-level semantic features from the semantic repository and seamlessly integrate low-level visual features and high-level semantic features in short-term learning to effectively represent the query in a single retrieval session. The high-level semantic features are dynamically updated based on users’ query concept and therefore represent the image’s semantic concept more accurately. Our extensive experimental results demonstrate that the proposed system outperforms its seven state-of-the-art peer systems in terms of retrieval precision and storage space on a large scale imagery database.     

Relevance Feedback and Learning in Content-Based Image Search

A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.     

Comparative analysis of relevance feedback methods based on two user studies

Rigorous analysis of user interest in web documents is essential for the development of recommender systems. This paper investigates the relationship between the implicit parameters and user explicit rating during their search and reading tasks. The objective of this paper is therefore three-fold: firstly, the paper identifies the implicit parameters which are statistically correlated with the user explicit rating through user study 1. These parameters are used to develop a predictive model which can be used to represent users’ perceived relevance of documents. Secondly, it investigates the reliability and validity of the predictive model by comparing it with eye gaze during a reading task through user study 2. Our findings suggest that there is no significant difference between the predictive model based on implicit indicators and eye gaze within the context examined. Thirdly, we measured the consistency of user explicit rating in both studies and found significant consistency in user explicit rating of document relevance and interest level which further validates the predictive model. We envisage that the results presented in this paper can help to develop recommender and personalised systems for recommending documents to users based on their previous interaction with the system.     

基于灰色系统理论的多特征相关反馈图像检索

探讨了灰色系统理论在相关反馈图像检索中的应用,提出了一种基于灰色系统理论的相关反馈图像检索算法,动态地更新查询向量,从而使图像检索结果与人的主观感知更加接近,具有自适应性。实验结果表明,文中的方法是很有效的。     

一种基于SVM的相关反馈图像检索算法

相关反馈技术是近年来图像检索中的研究热点,以MPEG-7的边缘直方图作为图像特征,以支持向蕈机(SvM)为分类器,提出一种新的相关反馈算法.在每次反馈中对用户标记的相关样本进行学习,用历次返回的结果更新训练样本集,建立SVM分类器模型,并根据模型进行检索.还对不同核函数的SVM进行了对比,得出RBF核函数的SVM有较高的检索精度.使用由10000幅图像组成的图像库进行实验,结果表明,算法可有效地检索出更多的相关图像,并且在有限训练样本情况下具有良好的泛化能力.     

An adaptive technique for content-based image retrieval

We discuss an adaptive approach towards Content-Based Image Retrieval. It is based on the Ostensive Model of developing information needs—a special kind of relevance feedback model that learns from implicit user feedback and adds a temporal notion to relevance. The ostensive approach supports content-assisted browsing through visualising the interaction by adding user-selected images to a browsing path, which ends with a set of system recommendations. The suggestions are based on an adaptive query learning scheme, in which the query is learnt from previously selected images. Our approach is an adaptation of the original Ostensive Model based on textual features only, to include content-based features to characterise images. In the proposed scheme textual and colour features are combined using the Dempster-Shafer theory of evidence combination. Results from a user-centred, work-task oriented evaluation show that the ostensive interface is preferred over a traditional interface with manual query facilities. This is due to its ability to adapt to the user's need, its intuitiveness and the fluid way in which it operates. Studying and comparing the nature of the underlying information need, it emerges that our approach elicits changes in the user's need based on the interaction, and is successful in adapting the retrieval to match the changes. In addition, a preliminary study of the retrieval performance of the ostensive relevance feedback scheme shows that it can outperform a standard relevance feedback strategy in terms of image recall in category search.     

A unified log-based relevance feedback scheme for image retrieval

Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). In the past, most research efforts in this field have focused on designing effective algorithms for traditional relevance feedback. Given that a CBIR system can collect and store users' relevance feedback information in a history log, an image retrieval system should be able to take advantage of the log data of users' feedback to enhance its retrieval performance. In this paper, we propose a unified framework for log-based relevance feedback that integrates the log of feedback data into the traditional relevance feedback schemes to learn effectively the correlation between low-level image features and high-level concepts. Given the error-prone nature of log data, we present a novel learning technique, named soft label support vector machine, to tackle the noisy data problem. Extensive experiments are designed and conducted to evaluate the proposed algorithms based on the COREL image data set. The promising experimental results validate the effectiveness of our log-based relevance feedback scheme empirically.     

Effective image retrieval techniques based on novel salient region segmentation and relevance feedback

This paper proposes an effective region-based image retrieval technique based on novel salient region segmentation and relevance feedback. With a good and fast segmentation technique, our system achieves an on-the-fly segmentation capability, which enables users to select particular regions for matching and feedbacks without waiting for image segmentation. Therefore, we adopt a relatively simple feedback schemes to derive the intent of the user. The experimental results show that the system performance is greatly improved with this capability. Furthermore, a Quick-match algorithm is also presented in this paper. The mechanism of the Quick-match algorithm is to exclude from distance computation regions that are of low possibility to be the top-Mmatches. This algorithm excludes most of regions from distance computation and therefore greatly cuts down the turnaround time of the retrieval with slightly degradation of precision.     

一种基于内容的图像检索界面

基于内容和对象的图像压缩和检索是下一代的图像处理技术,具有较广阔的应用前景。目前该领域的研究主要从设计方便、快捷的用户查询界面和发展图像数据库检索技术两方面展开。为此,该文提出了一种基于内容的图像检索用户界面的设计方法来满足用户复杂的检索要求。在图像的检索过程中,通过用户组合图标的方法来描述检索要求,同时将图像的颜色和空间信息相结合进行图像查询,并借助用户的反馈信息实现系统的自学习功能,最终逐步提高系统图像检索的速度和准确性。     

Active concept learning in image databases.

Concept learning in content-based image retrieval systems is a challenging task. This paper presents an active concept learning approach based on the mixture model to deal with the two basic aspects of a database system: the changing (image insertion or removal) nature of a database and user queries. To achieve concept learning, we a) propose a new user directed semi-supervised expectation-maximization algorithm for mixture parameter estimation, and b) develop a novel model selection method based on Bayesian analysis that evaluates the consistency of hypothesized models with the available information. The analysis of exploitation versus exploration in the search space helps to find the optimal model efficiently. Our concept knowledge transduction approach is able to deal with the cases of image insertion and query images being outside the database. The system handles the situation where users may mislabel images during relevance feedback. Experimental results on Corel database show the efficacy of our active concept learning approach and the improvement in retrieval performance by concept transduction.     

基于索引和相关反馈的图像检索研究

在基于图像内容的图像检索(CBIR)系统中,搜索引擎检索图像类似于按照相似标准来查询图像,它应该有足够快的速度并且有较高的检索准确率.索引用来提高系统响应,而相关反馈用于帮助提高检索准确率.在本文中,主要说明基于人感知的相似性度量,以及讨论综合相关反馈的索引方案.该索引方案通过分析特征熵而得出的主从键,而相关反馈是根据Mann-Whitnev检验而提出的,该检验通常用来识别来自同一搜索集中相关图像和不相关图像之间不同特征,并利用不同特征的特点提高检索性能.相关反馈方案针对两不同相似标准来执行,检验判定了这个方法的有效性.最后,把索引机制和相关反馈机制结合起来建立搜索引擎.     

Fuzzy SVM for content-based image retrieval: a pseudo-label support vector machine framework

Conventional relevance feedback in content-based image retrieval (CBIR) systems uses only the labeled images for learning. Image labeling, however, is a time-consuming task and users are often unwilling to label too many images during the feedback process. This gives rise to the small sample problem where learning from a small number of training samples restricts the retrieval performance. To address this problem, we propose a technique based on the concept of pseudo-labeling in order to enlarge the training data set. As the name implies, a pseudo-labeled image is an image not labeled explicitly by the users, but estimated using a fuzzy rule. Therefore, it contains a certain degree of uncertainty or fuzziness in its class information. Fuzzy support vector machine (FSVM), an extended version of SVM, takes into account the fuzzy nature of some training samples during its training. In order to exploit the advantages of pseudo-labeling, active learning and the structure of FSVM, we develop a unified framework called pseudo-label fuzzy support vector machine (PLFSVM) to perform content-based image retrieval. Experimental results based on a database of 10,000 images demonstrate the effectiveness of the proposed method.