Visual information retrieval system via content-based approach

In this paper a content-based image retrieval method that can search large image databases efficiently by color, texture, and shape content is proposed. Quantized RGB histograms and the dominant triple (hue, saturation, and value), which are extracted from quantized HSV joint histogram in the local image region, are used for representing global/local color information in the image. Entropy and maximum entry from co-occurrence matrices are used for texture information and edge angle histogram is used for representing shape information. Relevance feedback approach, which has coupled proposed features, is used for obtaining better retrieval accuracy. A new indexing method that supports fast retrieval in large image databases is also presented. Tree structures constructed by k-means algorithm, along with the idea of triangle inequality, eliminate candidate images for similarity calculation between query image and each database image. We find that the proposed method reduces calculation up to average 92.2 percent of the images from direct comparison.     

A general shape context framework for object identification

A general shape context framework is proposed for object/image retrieval in occluded and cluttered environment with hundreds of models as the potential matches of an input. The approach is general since it does not require separation of input objects from complex background. It works by first extracting consistent and structurally unique local neighborhood information from inputs or models, and then voting on the optimal matches. Its performance degrades gracefully with respect to the amount of structural information that is being occluded or lost. The local neighborhood information applicable to the system can be shape, color, texture feature, etc. Currently, we employ shape information only. The mechanism of voting is based on a novel hyper cube based indexing structure, and driven by dynamic programming. The proposed concepts have been tested on database with thousands of images. Very encouraging results have been obtained.     

Structural feature indexing for retrieval of partially visible shapes

Efficient and robust information retrieval from large image databases is an essential functionality for the reuse, manipulation, and editing of multimedia documents. Structural feature indexing is a potential approach to efficient shape retrieval from large image databases, but the indexing is sensitive to noise, scales of observation, and local shape deformations. It has now been confirmed that efficiency of classification and robustness against noise and local shape transformations can be improved by the feature indexing approach incorporating shape feature generation techniques (Nishida, Comput. Vision Image Understanding 73 (1) (1999) 121-136). In this paper, based on this approach, an efficient, robust method is presented for retrieval of model shapes that have parts similar to the query shape presented to the image database. The effectiveness is confirmed by experimental trials with a large database of boundary contours obtained from real images, and is validated by systematically designed experiments with a large number of synthetic data.     

Thesaurus-based 3D Object Retrieval with Part-in-Whole Matching

Research in content-based 3D retrieval has already started, and several approaches have been proposed which use in different manner a similarity assessment to match the shape of the query against the shape of the objects in the database. However, the success of these solutions are far from the success obtained by their textual counterparts. A major drawback of most existing 3D retrieval solutions is their inability to support partial queries, that is, a query which does not need to be formulated by specifying a whole query shape, but just a part of it, for example a detail of its overall shape, just like documents are retrieved by specifying words and not whole texts. Recently, researchers have focused their investigation on 3D retrieval which is solved by partial shape matching. However, at the extent of our knowledge, there is still no 3D search engine that provides an indexing of the 3D models based on all the interesting subparts of the models. In this paper we present a novel approach to 3D shape retrieval that uses a collection-aware shape decomposition combined with a shape thesaurus and inverted indexes to describe and retrieve 3D models using part-in-whole matching. The proposed method clusters similar segments obtained trough a multilevel decomposition of models, constructing from such partition the shape thesaurus. Then, to retrieve a model containing a sub-part similar to a given query, instead of looking on a large set of subparts or executing partial matching between the query and all models in the collection, we just perform a fast global matching between the query and the few entries in the thesaurus. With this technique we overcame the time complexity problems associated with partial queries in large collections.     

Fast Retrieval of Isolated Visual Shapes

Similarity-based retrieval from databases of isolated visual shapes has become an important information retrieval problem. The goal of the current work is to achieve high retrieval speed with reasonable retrieval effectiveness, and support for partial and occluded shape queries. In the proposed method, histograms of local shape parts are coded as index vectors. To increase retrieval accuracy, a rich set of parts at all scales of the shape is used; specifically, the parts are defined as connected sequences of regions in curvature scale space. To increase efficiency, structural indexing is used to compare the index vectors of the query and database shapes. In experimental evaluations, the method retrieved at least one similar shape in the top three retrieved items 99–100% of the time, depending on the database. Average retrieval times ranged from 0.7 ms on a 131-shape database to 7 ms on a 1310-shape database. The method is thus suitable for fast, approximate shape retrieval in comparison with more accurate but more costly structural matching.     

Quantized ranking for permutation-based indexing

The K-Nearest Neighbor (K-NN) search problem is the way to find the K closest and most similar objects to a given query. The K-NN is essential for many applications such as information retrieval and visualization, machine learning and data mining. The exponential growth of data imposes to find approximate approaches to this problem. Permutation-based indexing is one of the most recent techniques for approximate similarity search. Objects are represented by permutation lists ordering their distances to a set of selected reference objects, following the idea that two neighboring objects have the same surrounding. In this paper, we propose a novel quantized representation of permutation lists with its related data structure for effective retrieval on single and multicore architectures. Our novel permutation-based indexing strategy is built to be fast, memory efficient and scalable. This is experimentally demonstrated in comparison to existing proposals using several large-scale datasets of millions of documents and of different dimensions.     

Real-Time Motion Trajectory-Based Indexing and Retrieval of Video Sequences

This paper presents a novel motion trajectory-based compact indexing and efficient retrieval mechanism for video sequences. Assuming trajectory information is already available, we represent trajectories as temporal ordering of subtrajectories. This approach solves the problem of trajectory representation when only partial trajectory information is available due to occlusion. It is achieved by a hypothesis testing-based method applied to curvature data computed from trajectories. The subtrajectories are then represented by their principal component analysis (PCA) coefficients for optimally compact representation. Different techniques are integrated to index and retrieve subtrajectories, including PCA, spectral clustering, and string matching. We assume a query by example mechanism where an example trajectory is presented to the system and the search system returns a ranked list of most similar items in the dataset. Experiments based on datasets obtained from University of California at Irvine's KDD archives and Columbia University's DVMM group demonstrate the superiority of our proposed PCA-based approaches in terms of indexing and retrieval times and precision recall ratios, when compared to other techniques in the literature     

Signature Detection and Matching for Document Image Retrieval

As one of the most pervasive methods of individual identification and document authentication, signatures present convincing evidence and provide an important form of indexing for effective document image processing and retrieval in a broad range of applications. However, detection and segmentation of free-form objects such as signatures from clustered background is currently an open document analysis problem. In this paper, we focus on two fundamental problems in signature-based document image retrieval. First, we propose a novel multiscale approach to jointly detecting and segmenting signatures from document images. Rather than focusing on local features that typically have large variations, our approach captures the structural saliency using a signature production model and computes the dynamic curvature of 2D contour fragments over multiple scales. This detection framework is general and computationally tractable. Second, we treat the problem of signature retrieval in the unconstrained setting of translation, scale, and rotation invariant nonrigid shape matching. We propose two novel measures of shape dissimilarity based on anisotropic scaling and registration residual error and present a supervised learning framework for combining complementary shape information from different dissimilarity metrics using LDA. We quantitatively study state-of-the-art shape representations, shape matching algorithms, measures of dissimilarity, and the use of multiple instances as query in document image retrieval. We further demonstrate our matching techniques in offline signature verification. Extensive experiments using large real-world collections of English and Arabic machine-printed and handwritten documents demonstrate the excellent performance of our approaches.     

Intelligent multimedia data: data + indices + inference

In this paper, we present a novel approach for multimedia data indexing and retrieval that is machine independent and highly flexible for sharing multimedia data across applications. Traditional multimedia data indexing and retrieval problems have been attacked using the central data server as the main focus, and most of the indexing and query-processing for retrieval are highly application dependent. This precludes the use of created indices and query processing mechanisms for multimedia data which, in general, have a wide variety of uses across applications. The approach proposed in this paper addresses three issues: 1. multimedia data indexing; 2. inference or query processing; and 3. combining indices and inference or query mechanism with the data to facilitate machine independence in retrieval and query processing. We emphasize the third issue, as typically multimedia data are huge in size and requires intra-data indexing. We describe how the proposed approach addresses various problems faced by the application developers in indexing and retrieval of multimedia data. Finally, we present two applications developed based on the proposed approach: video indexing; and video content authorization for presentation.     

Hough Pyramid Matching: Speeded-Up Geometry Re-ranking for Large Scale Image Retrieval

Exploiting local feature shape has made geometry indexing possible, but at a high cost of index space, while a sequential spatial verification and re-ranking stage is still indispensable for large scale image retrieval. In this work we investigate an accelerated approach for the latter problem. We develop a simple spatial matching model inspired by Hough voting in the transformation space, where votes arise from single feature correspondences. Using a histogram pyramid, we effectively compute pair-wise affinities of correspondences without ever enumerating all pairs. Our Hough pyramid matching algorithm is linear in the number of correspondences and allows for multiple matching surfaces or non-rigid objects under one-to-one mapping. We achieve re-ranking one order of magnitude more images at the same query time with superior performance compared to state of the art methods, while requiring the same index space. We show that soft assignment is compatible with this matching scheme, preserving one-to-one mapping and further increasing performance.     

Cross-media retrieval using query dependent search methods

The content-based cross-media retrieval is a new type of multimedia retrieval in which the media types of query examples and the returned results can be different. In order to learn the semantic correlations among multimedia objects of different modalities, the heterogeneous multimedia objects are analyzed in the form of multimedia document (MMD), which is a set of multimedia objects that are of different media types but carry the same semantics. We first construct an MMD semi-semantic graph (MMDSSG) by jointly analyzing the heterogeneous multimedia data. After that, cross-media indexing space (CMIS) is constructed. For each query, the optimal dimension of CMIS is automatically determined and the cross-media retrieval is performed on a per-query basis. By doing this, the most appropriate retrieval approach for each query is selected, i.e. different search methods are used for different queries. The query dependent search methods make cross-media retrieval performance not only accurate but also stable. We also propose different learning methods of relevance feedback (RF) to improve the performance. Experiment is encouraging and validates the proposed methods.     

Unified framework for representing and ranking

In the database retrieval and nearest neighbor classification tasks, the two basic problems are to represent the query and database objects, and to learn the ranking scores of the database objects to the query. Many studies have been conducted for the representation learning and the ranking score learning problems, however, they are always learned independently from each other. In this paper, we argue that there are some inner relationships between the representation and ranking of database objects, and try to investigate their relationships by learning them in a unified way. To this end, we proposed the Unified framework for Representation and Ranking (UR²) of objects for the database retrieval and nearest neighbor classification tasks. The learning of representation parameter and the ranking scores are modeled within one single unified objective function. The objective function is optimized alternately with regard to representation parameter and the ranking scores. Based on the optimization results, iterative algorithms are developed to learn the representation parameter and the ranking scores on a unified way. Moreover, with two different formulas of representation (feature selection and subspace learning), we give two versions of UR². The proposed algorithms are tested on two challenging tasks – MRI image based brain tumor retrieval and nearest neighbor classification based protein identification. The experiments show the advantage of the proposed unified framework over the state-of-the-art independent representation and ranking methods.     

Symbolic picture retrieval by relative–metric spatial relations

Symbolic pictures can be used for iconic indexing, spatial reasoning, and similarity retrieval in the design of intelligent image database systems. [S. K. Chang, C. W. Yan, Donald C. Dimitroff, and Timothy Arndt, IEEE Trans. Software Engineering 1988, 14, 681–688; S.‐K. Chang, Principles of Pictorial Information Systems Design, Prentice‐Hall, New York, 1989.] However, previous approaches to designing such systems usually ignore relative‐metric information on symbolic pictures and cause several deficiencies in indexing, spatial reasoning, and retrieval. In our approach, we extract relative‐metric information from symbolic pictures and use such information to help establish indexes based on an improvement from a minimal perfect hashing scheme. As a result, more accurate picture retrieval can be achieved through our indexing mechanism. Capabilities in spatial reasoning and query representation/processing are also improved. By utilizing relative‐metric spatial relations, an image database system becomes more flexible and intelligent. ©2000 John Wiley & Sons, Inc.     

Graph-based shape indexing

Graphs have become growingly important in representing shapes in computer vision. Given a query graph, it is essential to retrieve similar database graphs efficiently from a large database. In this paper, we present a graph-based indexing technique which overcomes significant drawbacks of the previous work (Demirci et al. in Comput Vis Image Underst 110(3):312–325, 2008) using a recently developed theorem from the domain of matrix analysis. Our technique starts by representing the topological structure of a graph in a vector space. As done in the previous work, the topological structure of a graph is constructed using its Laplacian spectra. However, unlike the previous approach, which represents all sugraphs of a database graph in the vector space to account for local similarity, a database graph in the proposed framework is represented as a single vector. By performing a range search around the query, the proposed indexing technique returns a set with both partial and global similarity. Empirical evaluation of the algorithm on an extensive set of retrieval trials including a comparison with the previous approach in both 2D and 3D demonstrates the effectiveness, efficiency, and robustness of the overall approach.     

Image database design based on 9D-SPA representation for spatial relations

Spatial relationships between objects are important features for designing a content-based image retrieval system. We propose a new scheme, called 9D-SPA representation, for encoding the spatial relations in an image. With this representation, important functions of intelligent image database systems such as visualization, browsing, spatial reasoning, iconic indexing, and similarity retrieval can be easily achieved. The capability of discriminating images based on 9D-SPA representation is much more powerful than any spatial representation method based on minimum bounding rectangles or centroids of objects. The similarity measures using 9D-SPA representation provide a wide range of fuzzy matching capability in similarity retrieval to meet different user's requirements. Experimental results showed that our system is very effective in terms of recall and precision. In addition, the 9D-SPA representation can be incorporated into a two-level index structure to help reduce the search space of each query processing. The experimental results also demonstrated that, on average, only 0.1254 percent /spl sim/ 1.6829 percent of symbolic pictures (depending on various degrees of similarity) were accessed per query in an image database containing 50,000 symbolic pictures.     

一种基于Lucene改进的全文检索工具包

基于Lucene实现了一种改进的全文检索引擎工具包ELucene。它引入了索引配置文件,可针对不同应用背景来灵活定制索引的细节;提供了定时自动更新索引的功能;通过动态多态机制实现了支持多种索引数据源的功能;ELucene内部设计了引擎基础对象类,并以静态对象的方式运行来避免频繁读取索引文件带来的性能损失。面向检索,提供了检索请求类和检索响应类来分别封装用户的查询要求和查询结果集,并设计实现了一些实用的查询输入和输出处理的方法。基于ELucene的元数据搜索系统已成功应用到“国家科学数据共享工程：地球系统科学数据共享网”中。