文本提取和相似反馈的互联网图像检索研究

使用基于文本的互联网图像检索技术是互联网图像检索最实用的方式,也对其他方式的互联网图像检索有重要辅助作用,但如何利用周边文本来对图像进行准确描述一直是一个难题。利用TFIDF为基础提出了一个基于句法和文本重要性分类的图像关键词权重计算方法,并尝试通过图像的相似性因素作为反馈进一步优化搜索结果,为用户返回最贴切的搜索结果。     

A scalable approach for content based image retrieval in cloud datacenter

The emergence of cloud datacenters enhances the capability of online data storage. Since massive data is stored in datacenters, it is necessary to effectively locate and access interest data in such a distributed system. However, traditional search techniques only allow users to search images over exact-match keywords through a centralized index. These techniques cannot satisfy the requirements of content based image retrieval (CBIR). In this paper, we propose a scalable image retrieval framework which can efficiently support content similarity search and semantic search in the distributed environment. Its key idea is to integrate image feature vectors into distributed hash tables (DHTs) by exploiting the property of locality sensitive hashing (LSH). Thus, images with similar content are most likely gathered into the same node without the knowledge of any global information. For searching semantically close images, the relevance feedback is adopted in our system to overcome the gap between low-level features and high-level features. We show that our approach yields high recall rate with good load balance and only requires a few number of hops.     

Building Visual Vocabulary for Image Indexation and Query Formulation

With advances in digital imaging, the amount of digital images will increase tremendously. To locate relevant images in a large collection of images presents a challenging and genuine problem for content-based retrieval research. This paper presents a novel framework called visual keywords for image indexation and query formulation. Visual keywords are flexible and intuitive visual prototypes specified perceptually from sample domain images. A visual content is described and indexed by flexible spatial aggregation of the soft presence of visual keywords. A new query method based on visual constraints is also proposed to allow direct and explicit content specification. Last but not least, we have developed a digital album prototype to demonstrate query and retrieval on both home photos and stock photos based on visual keywords.     

Effective image annotation for searches using multilevel semantics

There is an increasing need for automatic image annotation tools to enable effective image searching in digital libraries. In this paper, we present a novel probabilistic model for image annotation based on content-based image retrieval techniques and statistical analysis. One key difficulty in applying statistical methods to the annotation of images is that the number of manually labeled images used to train the methods is normally insufficient. Numerous keywords cannot be correctly assigned to appropriate images due to lacking or missing information in the labeled image databases. To deal with this challenging problem, we also propose an enhanced model in which the annotated keywords of a new image are defined in terms of their similarity at different semantic levels, including the image level, keyword level, and concept level. To avoid missing some relevant keywords, the model labels the keywords with the same concepts as the new image. Our experimental results show that the proposed models are effective for annotating images that have different qualities of training data.     

Automatic Annotation and Retrieval of Images

Although a variety of techniques have been developed for content-based image retrieval (CBIR), automatic image retrieval by semantics still remains a challenging problem. We propose a novel approach for semantics-based image annotation and retrieval. Our approach is based on the monotonic tree model. The branches of the monotonic tree of an image, termed as structural elements, are classified and clustered based on their low level features such as color, spatial location, coarseness, and shape. Each cluster corresponds to some semantic feature. The category keywords indicating the semantic features are automatically annotated to the images. Based on the semantic features extracted from images, high-level (semantics-based) querying and browsing of images can be achieved. We apply our scheme to analyze scenery features. Experiments show that semantic features, such as sky, building, trees, water wave, placid water, and ground, can be effectively retrieved and located in images.     

Developing an object‐oriented framework for content‐based image retrieval

Content‐based image retrieval (CBIR) is a process of retrieving images from an image database by exploiting the content of the images (typically the querying of an image). CBIR avoids many problems associated with traditional ways of retrieving images by keywords. Thus, a growing interest in the area of CBIR has been established in recent years. In this paper, a novel object‐oriented framework (CBIRFrame) is built for CBIR applications development. We discuss the motivations for CBIRFrame before discussing its design in detail. Two applications of CBIRFrame are also briefly discussed to show the effectiveness of applying CBIRFrame to real applications. Finally, we outline the possible uses of the design of CBIRFrame for other types of domains, such as content‐based retrieval of video clips. Copyright © 2003 John Wiley & Sons, Ltd.     

Color indexing for efficient image retrieval

Content based image retrieval is an active area of research. Many approaches have been proposed to retrieve images based on matching of some features derived from the image content. Color is an important feature of image content. The problem with many traditional matching-based retrieval methods is that the search time for retrieving similar images for a given query image increases linearly with the size of the image database. We present an efficient color indexing scheme for similarity-based retrieval which has a search time that increases logarithmically with the database size.In our approach, the color features are extracted automatically using a color clustering algorithm. Then the cluster centroids are used as representatives of the images in 3-dimensional color space and are indexed using a spatial indexing method that usesR-tree. The worst case search time complexity of this approach isOn _q log(N^* n_avg)), whereN is the number of images in the database, andn _q andn _avg are the number of colors in the query image and the average number of colors per image in the database respectively. We present the experimental results for the proposed approach on two databases consisting of 337 Trademark images and 200 Flag images.     

Web-based image indexing and retrieval in JPEG compressed domain

As the majority of content-based image retrieval systems operate on full images in pixel domain, decompression is a prerequisite for the retrieval of compressed images. To provide a possible on-line indexing and retrieval technique for those jpg image files, we propose a novel pseudo-pixel extraction algorithm to bridge the gap between the existing image indexing technology, developed in the pixel domain, and the fact that an increasing number of images stored on the Web are already compressed by JPEG at the source. Further, we describe our Web-based image retrieval system, WEBimager, by using the proposed algorithm to provide a prototype visual information system toward automatic management, indexing, and retrieval of compressed images available on the Internet. This provides users with efficient tools to search the Web for compressed images and establish a database or a collection of special images to their interests. Experiments using texture- and colour-based indexing techniques support the idea that the proposed algorithm achieves significantly better results in terms of computing cost than their full decompression or partial decompression counterparts. This technology will help control the explosion of media-rich content by offering users a powerful automated image indexing and retrieval tool for compressed images on the Web.J. Jiang: Contacting author     

Active concept learning in image databases.

Concept learning in content-based image retrieval systems is a challenging task. This paper presents an active concept learning approach based on the mixture model to deal with the two basic aspects of a database system: the changing (image insertion or removal) nature of a database and user queries. To achieve concept learning, we a) propose a new user directed semi-supervised expectation-maximization algorithm for mixture parameter estimation, and b) develop a novel model selection method based on Bayesian analysis that evaluates the consistency of hypothesized models with the available information. The analysis of exploitation versus exploration in the search space helps to find the optimal model efficiently. Our concept knowledge transduction approach is able to deal with the cases of image insertion and query images being outside the database. The system handles the situation where users may mislabel images during relevance feedback. Experimental results on Corel database show the efficacy of our active concept learning approach and the improvement in retrieval performance by concept transduction.     

基于文本与语义相关性分析的图像检索

为了更加有效地检索到符合用户复杂语义需求的图像，提出一种基于文本描述与语义相关性分析的图像检索算法。该方法将图像检索分为两步：基于文本语义相关性分析的图像检索和基于SIFT特征的相似图像扩展检索。根据自然语言处理技术分析得到用户文本需求中的关键词及其语义关联，在选定图像库中通过语义相关性分析得到“种子”图像；接下来在图像扩展检索中，采用基于SIFT特征的相似图像检索，利用之前得到的“种子”图像作为查询条件，在网络图像库中进行扩展检索，并在结果集上根据两次检索的图像相似度进行排序输出，最终得到更加丰富有效的图像检索结果。为了证明算法的有效性，在标准数据集Corel5K和网络数据集Deriantart8K上完成了多组实验，实验结果证明该方法能够得到较为精确地符合用户语义要求的图像检索结果，并且通过扩展算法可以得到更加丰富的检索结果。     

Photograph retrieval and classification by visual keywords and thesaurus

As we collect more digital images with the advent of digital cameras, we need effective content-based search and categorization functions. In this paper, we propose a novel notion of visual keywords to describe and compare digital visual contents. Visual keywords are visual prototypes extracted from a visual content domain with semantics labels. They can be further abstracted to form visual thesaurus. An image is indexed as a spatial distribution of visual keywords. Both retrieval and classification evaluation tasks on professional natural scene photographs have demonstrated the usefulness of this new methodology. Joo-Hwee Lim: He received his B.S. (Hons I) and M.S. degrees in Computer Science from the National University of Singapore in 1989 and 1991 respectively. He has joined Kent Ridge Digital Labs (KRDL), Singapore since Oct 1990 and is currently an associate research staff of KRDL. He has published widely in his areas of research interests which include content-based processing, pattern recognition, and neural networks.     

An evaluation of semantic fisheye views for opportunistic search in an annotated image collection

Visual interfaces are potentially powerful tools for users to explore a representation of a collection and opportunistically discover information that will guide them toward relevant documents. Semantic fisheye views (SFEVs) are focus + context visualization techniques that manage visual complexity by selectively emphasizing and increasing the detail of information related to the users focus and deemphasizing or filtering less important information.In this paper we describe a prototype for visualizing an annotated image collection and an experiment to compare the effectiveness of two distinctly different SFEVs for a complex opportunistic search task. The first SFEV calculates relevance based on keyword-content similarity and the second based on conceptual relationships between images derived using WordNet. The results of the experiment suggest that semantic-guided search is significantly more effective than similarity-guided search for discovering and using domain knowledge in a collection.     

Filter Image Browsing: Interactive Image Retrieval by Using Database Overviews

Human-computer interaction is a decisive factor in effective content-based access to large image repositories. In current image retrieval systems the user refines his query by selecting example images from a relevance ranking. Since the top ranked images are all similar, user feedback often results in rearrangement of the presented images only.For better incorporation of user interaction in the retrieval process, we have developed the Filter Image Browsing method. It also uses feedback through image selection. However, it is based on differences between images rather than similarities. Filter Image Browsing presents overviews of relevant parts of the database to users. Through interaction users then zoom in on parts of the image collection. By repeatedly limiting the information space, the user quickly ends up with a small amount of relevant images. The method can easily be extended for the retrieval of multimedia objects.For evaluation of the Filter Image Browsing retrieval concept, a user simulation is applied to a pictorial database containing 10,000 images acquired from the World Wide Web by a search robot. The simulation incorporates uncertainty in the definition of the information need by users. Results show Filter Image Browsing outperforms plain interactive similarity ranking in required effort from the user. Also, the method produces predictable results for retrieval sessions, so that the user quickly knows if a successful session is possible at all. Furthermore, the simulations show the overview techniques are suited for applications such as hand-held devices where screen space is limited.     

Integration of Image Matching and Classification for Multimedia Navigation

With the recent explosive growth in the volume of images on the World-Wide Web, it has become increasingly difficult to search for images of interests. The classification of images helps users to access a large image collection efficiently. Classification reduces search space by filtering out unrelated images. Classification also allows for more user-friendly interfaces: users can better visualize easily result space by browsing the representative images of the candidates. In this paper, we present a technique for image classification based on color, shape and composition using the primary objects. We apply this classification technique in image matching for image retrieval on the Web. Our experimental results show that this approach can maintain 73% of recall by searching only 24% of the whole data set. We also show how we apply such technique to assist users in navigation.     

Improving image tags by exploiting web search results

Automatic image tagging automatically assigns image with semantic keywords called tags, which significantly facilitates image search and organization. Most of present image tagging approaches are constrained by the training model learned from the training dataset, and moreover they have no exploitation on other type of web resource (e.g., web text documents). In this paper, we proposed a search based image tagging algorithm (CTSTag), in which the result tags are derived from web search result. Specifically, it assigns the query image with a more comprehensive tag set derived from both web images and web text documents. First, a content-based image search technology is used to retrieve a set of visually similar images which are ranked by the semantic consistency values. Then, a set of relevant tags are derived from these top ranked images as the initial tag set. Second, a text-based search is used to retrieve other relevant web resources by using the initial tag set as the query. After the denoising process, the initial tag set is expanded with other tags mined from the text-based search result. Then, an probability flow measure method is proposed to estimate the probabilities of the expanded tags. Finally, all the tags are refined using the Random Walk with Restart (RWR) method and the top ones are assigned to the query images. Experiments on NUS-WIDE dataset show not only the performance of the proposed algorithm but also the advantage of image retrieval and organization based on the result tags.     

Spoken query based word spotting in digitized Tamil documents

This paper presents an integrated approach to spot the spoken keywords in digitized Tamil documents by combining word image matching and spoken word recognition techniques. The work involves the segmentation of document images into words, creation of an index of keywords, and construction of word image hidden Markov model (HMM) and speech HMM for each keyword. The word image HMMs are constructed using seven dimensional profile and statistical moment features and used to recognize a segmented word image for possible inclusion of the keyword in the index. The spoken query word is recognized using the most likelihood of the speech HMMs using the 39 dimensional mel frequency cepstral coefficients derived from the speech samples of the keywords. The positional details of the search keyword obtained from the automatically updated index retrieve the relevant portion of text from the document during word spotting. The performance measures such as recall, precision, and F-measure are calculated for 40 test words from the four groups of literary documents to illustrate the ability of the proposed scheme and highlight its worthiness in the emerging multilingual information retrieval scenario.     

The Mosaic Test: measuring the effectiveness of colour-based image retrieval

A variety of content-based image retrieval systems exist which enable users to perform image retrieval based on colour content—i.e., colour-based image retrieval. For the production of media for use in television and film, colour-based image retrieval is useful for retrieving specifically coloured animations, graphics or videos from large databases (by comparing user queries to the colour content of extracted key frames). It is also useful to graphic artists creating realistic computer-generated imagery (CGI). Unfortunately, current methods for evaluating colour-based image retrieval systems have 2 major drawbacks. Firstly, the relevance of images retrieved during the task cannot be measured reliably. Secondly, existing methods do not account for the creative design activity known as reflection-in-action. Consequently, the development and application of novel and potentially more effective colour-based image retrieval approaches, better supporting the large number of users creating media for use in television and film productions, is not possible as their efficacy cannot be reliably measured and compared to existing technologies. As a solution to the problem, this paper introduces the Mosaic Test. The Mosaic Test is a user-based evaluation approach in which participants complete an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. In this paper, we introduce the Mosaic Test and report on a user evaluation. The findings of the study reveal that the Mosaic Test overcomes the 2 major drawbacks associated with existing evaluation methods and does not require expert participants.     

Towards Data-Adaptive and User-Adaptive Image Retrieval by Peer Indexing

Adaptation to the characteristics of specific images and the preferences of individual users is critical to the success of an image retrieval system but insufficiently addressed by the existing approaches. In this paper, we propose an elegant and effective approach to data-adaptive and user-adaptive image retrieval based on the idea of peer indexing—describing an image through semantically relevant peer images. Specifically, we associate each image with a two-level peer index that models the “data characteristics” of the image as well as the “user characteristics” of individual users with respect to this image. Based on two-level image peer indexes, a set of retrieval parameters including query vectors and similarity metric are optimized towards both data and user characteristics by applying the pseudo feedback strategy. A cooperative framework is proposed under which peer indexes and image visual features are integrated to facilitate data- and user-adaptive image retrieval. Simulation experiments conducted on real-world images have verified the effectiveness of our approach in a relatively restricted setting.     

个性化搜索引擎系统机制的研究

随着网络信息资源的迅速增加,个性化信息服务越来越成为信息检索领域中研究的热点,针对传统搜索引擎系统的缺点,提出了一种新型个性化搜索引擎系统的体系结构,并在此基础上给出了系统中个性化机制的相关算法,同时使用基于关键词的搜索,利用Web挖掘技术,在实现为不同用户提供不同检索结果的同时提高了个性化查询的精确度和速度,保证了全查率.