Browsing museum image collections on a multi-touch table

In this paper we present a multi-touch tabletop system for browsing image databases, conceived for museums and art gallery exhibitions. The system exploits an innovative image browsing paradigm and image retrieval functionalities to perform natural and intuitive user interaction: users can explore the image database by handling digital pictures by means of touch gestures or by the use of a predetermined set of physical objects; once one of these objects is placed on the table, it is automatically recognized and the associated function is triggered. The set of objects and the function associations can be dynamically configured. An innovative feature of our application is that users can interactively create and manipulate image clusters where images are grouped according to their pictorial similarity. This is achieved by placing one or more specific tangible objects on the table surface. The system has been evaluated on a collection of photos organized in groups according to the UNESCO picture categories. The usability tests, performed considering different user categories, show that users consider the application to be attractive and interesting.     

Inference and retrieval of facial images

Attempts have been made to extend SQL to work with multimedia databases. We are reserved on the representation ability of extended SQL to cope with the richness in content of multimedia data. In this paper we present an example of a multimedia database system, Computer Aided Facial Image Inference and Retrieval system (CAFIIR). The system stores and manages facial images and criminal records, providing necessary functions for crime identification. We would like to demonstrate some core techniques for multimedia database with CAFIIR system. Firstly, CAFIIR is a integrated system. Besides database management, there are image analysis, image composition, image aging, and report generation subsystems, providing means for problem solving. Secondly, the richness of multimedia data urges feature-based database for their management. CAFIIR is feature-based. A indexing mechanism,iconic index, has been proposed for indexing facial images using hierarchical self-organization neural network. The indexing method operates on complex feature measures and provides means for visual navigation. Thirdly, special retrieval methods for facial images have been developed, including visual browsing, similarity retrieval, free text retrieval and fuzzy retrieval.     

Building a user-centered semantic hierarchy in image databases

Recent development in the field of digital media technology has resulted in the generation of a huge number of images. Consequently, content-based image retrieval has emerged as an important area in multimedia computing. Research in human perception of image content suggests that the semantic cues play an important role in image retrieval. In this paper, we present a new paradigm to establish the semantics in image databases based on multi-user relevance feedback. Relevance feedback mechanism is one way to incorporate the users’ perception during image retrieval. By treating each feedback as a weak classifier and combining them together, we are able to capture the categories in the users’ mind and build a user-centered semantic hierarchy in the database to support semantic browsing and searching. We present an image retrieval system based on a city-landscape image database comprising of 3,009 images. We also compare our approach with other typical methods to organize an image database. Superior results have been achieved by the proposed framework.     

A survey of browsing models for content based image retrieval

The problem of content based image retrieval (CBIR) has traditionally been investigated within a framework that emphasises the explicit formulation of a query: users initiate an automated search for relevant images by submitting an image or draw a sketch that exemplifies their information need. Often, relevance feedback is incorporated as a post-retrieval step for optimising the way evidence from different visual features is combined. While this sustained methodological focus has helped CBIR to mature, it has also brought out its limitations more clearly: There is often little support for exploratory search and scaling to very large collections is problematic. Moreover, the assumption that users are always able to formulate an appropriate query is questionable. An effective, albeit much less studied, method of accessing image collections based on visual content is that of browsing. The aim of this survey paper is to provide a structured overview of the different models that have been explored over the last one to two decades, to highlight the particular challenges of the browsing approach and to focus attention on a few interesting issues that warrant more intense research.

Interactive browsing via diversified visual summarization for image search results

Presenting and browsing image search results play key roles in helping users to find desired images from search results. Most existing commercial image search engines present them, dependent on a ranked list. However, such a scheme suffers from at least two drawbacks: inconvenience for consumers to get an overview of the whole result, and high computation cost to find desired images from the list. In this paper, we introduce a novel search result summarization approach and exploit this approach to further propose an interactive browsing scheme. The main contribution of this paper includes: (1) a dynamic absorbing random walk to find diversified representatives for image search result summarization; (2) a local scaled visual similarity evaluation scheme between two images through inspecting the relation between each image and other images; and (3) an interactive browsing scheme, based on a tree structure for organizing the images obtained from the summarization approach, to enable users to intuitively and conveniently browse the image search results. Quantitative experimental results and user study demonstrate the effectiveness of the proposed summarization and browsing approaches.     

Visual information retrieval system via content-based approach

In this paper a content-based image retrieval method that can search large image databases efficiently by color, texture, and shape content is proposed. Quantized RGB histograms and the dominant triple (hue, saturation, and value), which are extracted from quantized HSV joint histogram in the local image region, are used for representing global/local color information in the image. Entropy and maximum entry from co-occurrence matrices are used for texture information and edge angle histogram is used for representing shape information. Relevance feedback approach, which has coupled proposed features, is used for obtaining better retrieval accuracy. A new indexing method that supports fast retrieval in large image databases is also presented. Tree structures constructed by k-means algorithm, along with the idea of triangle inequality, eliminate candidate images for similarity calculation between query image and each database image. We find that the proposed method reduces calculation up to average 92.2 percent of the images from direct comparison.     

Indexing chromatic and achromatic patterns for content-based colour image retrieval

In this paper, we present a method to represent achromatic and chromatic image signals independently for content-based image indexing and retrieval for image database applications. Starting from an opponent colour representation, human colour vision theories and modern digital signal processing technologies are applied to develop a compact and computationally efficient visual appearance model for coloured image patterns. We use the model to compute the statistics of achromatic and chromatic spatial patterns of colour images for indexing and content-based retrieval. Two types of colour images databases, one colour texture database and another photography colour image database are used to evaluate the performance of the developed method in content-based image indexing and retrieval. Experimental results are presented to show that the new method is superior or competitive to state-of-the-art content-based image indexing and retrieval techniques.     

Content-based indexing of multimedia databases

Content-based retrieval of multimedia database calls for content-based indexing techniques. Different from conventional databases, where data items are represented by a set of attributes of elementary data types, multimedia objects in multimedia databases are represented by a collection of features; similarity of object contents depends on context and frame of reference; and features of objects are characterized by multimodal feature measures. These lead to great challenges for content-based indexing. On the other hand, there are special requirements on content-based indexing: to support visual browsing, similarity retrieval, and fuzzy retrieval, nodes of the index should represent certain meaningful categories. That is to say that certain semantics must be added when performing indexing. ContIndex, the context-based indexing technique presented in this paper, is proposed to meet these challenges and special requirements. The indexing tree is formally defined by adapting a classification-tree concept. Horizontal links among nodes in the same level enhance the flexibility of the index. A special neural-network model, called Learning based on Experiences and Perspectives (FEP), has been developed to create node categories by fusing multimodal feature measures. It brings into the index the capability of self-organizing nodes with respect to certain context and frames of reference. An icon image is generated for each intermediate node to facilitate visual browsing. Algorithms have been developed to support multimedia object archival and retrieval using Contlndex     

一种改进的SVM相关反馈图像检索方法*

提出了一种改进的支持向量机SVM( Support Vector Machine) 的相关反馈图像检索方法。在这种方法的交互过程中, SVM 分类器不仅对本次反馈过程中用户所提交的标记的正例和反例样本进行学习, 还对历次反馈过程中的正例和反例样本进行学习, 并根据训练后的分类器进行检索。实验结果表明, 该方法在样本集非常小的情况下, 仍可以检索出较多的相关图像, 在有限训练样本情况下具有良好的推广能力。     

Efficient content-based image retrieval using Multiple Support Vector Machines Ensemble

With the evolution of digital technology, there has been a significant increase in the number of images stored in electronic format. These range from personal collections to medical and scientific images that are currently collected in large databases. Many users and organizations now can acquire large numbers of images and it has been very important to retrieve relevant multimedia resources and to effectively locate matching images in the large databases. In this context, content-based image retrieval systems (CBIR) have become very popular for browsing, searching and retrieving images from a large database of digital images with minimum human intervention. The research community are competing for more efficient and effective methods as CBIR systems may be heavily employed in serving time critical applications in scientific and medical domains. This paper proposes an extremely fast CBIR system which uses Multiple Support Vector Machines Ensemble. We have used Daubechies wavelet transformation for extracting the feature vectors of images. The reported test results are very promising. Using data mining techniques not only improved the efficiency of the CBIR systems, but they also improved the accuracy of the overall process.     

Web-based image indexing and retrieval in JPEG compressed domain

As the majority of content-based image retrieval systems operate on full images in pixel domain, decompression is a prerequisite for the retrieval of compressed images. To provide a possible on-line indexing and retrieval technique for those jpg image files, we propose a novel pseudo-pixel extraction algorithm to bridge the gap between the existing image indexing technology, developed in the pixel domain, and the fact that an increasing number of images stored on the Web are already compressed by JPEG at the source. Further, we describe our Web-based image retrieval system, WEBimager, by using the proposed algorithm to provide a prototype visual information system toward automatic management, indexing, and retrieval of compressed images available on the Internet. This provides users with efficient tools to search the Web for compressed images and establish a database or a collection of special images to their interests. Experiments using texture- and colour-based indexing techniques support the idea that the proposed algorithm achieves significantly better results in terms of computing cost than their full decompression or partial decompression counterparts. This technology will help control the explosion of media-rich content by offering users a powerful automated image indexing and retrieval tool for compressed images on the Web.J. Jiang: Contacting author     

Image Collection Organization and Its Application to Indexing, Browsing, Summarization, and Semantic Retrieval

In this paper, we present a new framework for organizing image collections into structures that can be used for indexing, browsing, retrieval and summarization. Instead of using tree-based techniques which are not suitable for images, we develop a new solution that is specifically designed for image collections. We consider both low-level image content and high-level semantics in an attempt to alleviate the semantic gap encountered by many systems. The fact that our model is based on a probabilistic framework makes it possible to combine it in a natural way with probabilistic techniques developed recently for image retrieval. The structure our model generates is applied for four purposes. The first is to provide retrieval module with an index, which allows it to improve retrieval time and accuracy, while the second is to provide users with a hierarchical browsing catalog that allows them to navigate the image collection by subject. This represents an additional step towards facilitating human-computer interaction in the context of image retrieval and navigation. The third aim is to provide users with a summarization of the general content of each class in the collection, and the fourth is a retrieval mechanism. Related issues such as relevance feedback and feature selection are also addressed. The experiments at the end of the paper show that the proposed framework yields some significant improvements     

Integration of Image Matching and Classification for Multimedia Navigation

With the recent explosive growth in the volume of images on the World-Wide Web, it has become increasingly difficult to search for images of interests. The classification of images helps users to access a large image collection efficiently. Classification reduces search space by filtering out unrelated images. Classification also allows for more user-friendly interfaces: users can better visualize easily result space by browsing the representative images of the candidates. In this paper, we present a technique for image classification based on color, shape and composition using the primary objects. We apply this classification technique in image matching for image retrieval on the Web. Our experimental results show that this approach can maintain 73% of recall by searching only 24% of the whole data set. We also show how we apply such technique to assist users in navigation.     

Applying logistic regression to relevance feedback in image retrieval systems

This paper deals with the problem of image retrieval from large image databases. A particularly interesting problem is the retrieval of all images which are similar to one in the user's mind, taking into account his/her feedback which is expressed as positive or negative preferences for the images that the system progressively shows during the search. Here we present a novel algorithm for the incorporation of user preferences in an image retrieval system based exclusively on the visual content of the image, which is stored as a vector of low-level features. The algorithm considers the probability of an image belonging to the set of those sought by the user, and models the logit of this probability as the output of a generalized linear model whose inputs are the low-level image features. The image database is ranked by the output of the model and shown to the user, who selects a few positive and negative samples, repeating the process in an iterative way until he/she is satisfied. The problem of the small sample size with respect to the number of features is solved by adjusting several partial generalized linear models and combining their relevance probabilities by means of an ordered averaged weighted operator. Experiments were made with 40 users and they exhibited good performance in finding a target image (4 iterations on average) in a database of about 4700 images. The mean number of positive and negative examples is of 4 and 6 per iteration. A clustering of users into sets also shows consistent patterns of behavior.     

Adaptive image segmentation for region-based object retrieval using generalized Hough transform

Finding an object inside a target image by querying multimedia data is desirable, but remains a challenge. The effectiveness of region-based representation for content-based image retrieval is extensively studied in the literature. One common weakness of region-based approaches is that perform detection using low level visual features within the region and the homogeneous image regions have little correspondence to the semantic objects. Thus, the retrieval results are often far from satisfactory. In addition, the performance is significantly affected by consistency in the segmented regions of the target object from the query and database images. Instead of solving these problems independently, this paper proposes region-based object retrieval using the generalized Hough transform (GHT) and adaptive image segmentation. The proposed approach has two phases. First, a learning phase identifies and stores stable parameters for segmenting each database image. In the retrieval phase, the adaptive image segmentation process is also performed to segment a query image into regions for retrieving visual objects inside database images through the GHT with a modified voting scheme to locate the target visual object under a certain affine transformation. The learned parameters make the segmentation results of query and database images more stable and consistent. Computer simulation results show that the proposed method gives good performance in terms of retrieval accuracy, robustness, and execution speed.     

Visualization and User-Modeling for Browsing Personal Photo Libraries

We present a user-centric system for visualization and layout for content-based image retrieval. Image features (visual and/or semantic) are used to display retrievals as thumbnails in a 2-D spatial layout or “configuration” which conveys all pair-wise mutual similarities. A graphical optimization technique is used to provide maximally uncluttered and informative layouts. Moreover, a novel subspace feature weighting technique can be used to modify 2-D layouts in a variety of context-dependent ways. An efficient computational technique for subspace weighting and re-estimation leads to a simple user-modeling framework whereby the system can learn to display query results based on layout examples (or relevance feedback) provided by the user. The resulting retrieval, browsing and visualization can adapt to the user's (time-varying) notions of content, context and preferences in style and interactive navigation. Monte Carlo simulations with machine-generated layouts as well as pilot user studies have demonstrated the ability of this framework to model or “mimic” users, by automatically generating layouts according to their preferences.     

A Statistical Framework for Image Category Search from a Mental Picture

Starting from a member of an image database designated the "query image," traditional image retrieval techniques, for example, search by visual similarity, allow one to locate additional instances of a target category residing in the database. However, in many cases, the query image or, more generally, the target category, resides only in the mind of the user as a set of subjective visual patterns, psychological impressions, or "mental pictures." Consequently, since image databases available today are often unstructured and lack reliable semantic annotations, it is often not obvious how to initiate a search session; this is the "page zero problem." We propose a new statistical framework based on relevance feedback to locate an instance of a semantic category in an unstructured image database with no semantic annotation. A search session is initiated from a random sample of images. At each retrieval round, the user is asked to select one image from among a set of displayed images-the one that is closest in his opinion to the target class. The matching is then "mental." Performance is measured by the number of iterations necessary to display an image which satisfies the user, at which point standard techniques can be employed to display other instances. Our core contribution is a Bayesian formulation which scales to large databases. The two key components are a response model which accounts for the user's subjective perception of similarity and a display algorithm which seeks to maximize the flow of information. Experiments with real users and two databases of 20,000 and 60,000 images demonstrate the efficiency of the search process.