Improving image retrieval by using spatial relations

In this paper we proposed the use of spatial relations as a way of improving annotation-based image retrieval. We analyzed different types of spatial relations and selected the most adequate ones for image retrieval. We developed an image comparison and retrieval method based on conceptual graphs, which incorporates spatial relations. Additionally, we proposed an alternative term-weighting scheme and explored the use of more than one sample image for retrieval using several late fusion techniques. Our methods were evaluated with a rich and complex image dataset, based on the 39 topics developed for the ImageCLEF 2008 photo retrieval task. Results show that: (i) incorporating spatial relations produces a significant increase in performance, (ii) the label weighting scheme we proposed obtains better results than other traditional schemes, and (iii) the combination of several sample images using late fusion produces an additional improvement in retrieval according to several metrics.     

Sketch-based 3D model retrieval utilizing adaptive view clustering and semantic information

Multimedia Tools and Applications - Searching for relevant 3D models based on hand-drawn sketches is both intuitive and important for many applications, such as sketch-based 3D modeling and...     

Effective proximity retrieval by ordering permutations

We introduce a new probabilistic proximity search algorithm for range and K-nearest neighbor (K-NN) searching in both coordinate and metric spaces. Although there exist solutions for these problems, they boil down to a linear scan when the space is intrinsically high-dimensional, as is the case in many pattern recognition tasks. This, for example, renders the K-NN approach to classification rather slow in large databases. Our novel idea is to predict closeness between elements according to how they order their distances towards a distinguished set of anchor objects. Each element in the space sorts the anchor objects from closest to farthest to it, and the similarity between orders turns out to be an excellent predictor of the closeness between the corresponding elements. We present extensive experiments comparing our method against state-of-the-art exact and approximate techniques, both in synthetic and real, metric and non-metric databases, measuring both CPU time and distance computations. The experiments demonstrate that our technique almost always improves upon the performance of alternative techniques, in some cases by a wide margin.     

Multilevel cognitive analysis in graphical retrieval of drawings

Structural representation of drawings and their fragments in the form of loaded graphs with spatial relations as well as an original tool of their analysis are discussed. Several levels of representation of graphic design-engineering data are proposed. Examples of analyzing 3D images of products, which are reconstructed by drawings, are given.     

Sketch-based 3D model retrieval by incorporating 2D-3D alignment

Sketch-based 3D model retrieval is very important for applications such as 3D modeling and recognition. In this paper, a sketch-based retrieval algorithm is proposed based on a 3D model feature named View Context and 2D relative shape context matching. To enhance the accuracy of 2D sketch-3D model correspondence as well as the retrieval performance, we propose to align a 3D model with a query 2D sketch before measuring their distance. First, we efficiently select some candidate views from a set of densely sampled views of the 3D model to align the sketch and the model based on their View Context similarities. Then, we compute the more accurate relative shape context distance between the sketch and every candidate view, and regard the minimum one as the sketch-model distance. To speed up retrieval, we precompute the View Context and relative shape context features of the sample views of all the 3D models in the database. Comparative and evaluative experiments based on hand-drawn and standard line drawing sketches demonstrate the effectiveness and robustness of our approach and it significantly outperforms several latest sketch-based retrieval algorithms.     

Modeling term proximity for probabilistic information retrieval models

Proximity among query terms has been found to be useful for improving retrieval performance. However, its application to classical probabilistic information retrieval models, such as Okapi’s BM25, remains a challenging research problem. In this paper, we propose to improve the classical BM25 model by utilizing the term proximity evidence. Four novel methods, namely a window-based N-gram Counting method, Survival Analysis over different statistics, including the Poisson process, an exponential distribution and an empirical function, are proposed to model the proximity between query terms. Through extensive experiments on standard TREC collections, our proposed proximity-based BM25 model, called BM25P, is compared to strong state-of-the-art evaluation baselines, including the original unigram BM25 model, the Markov Random Field model, and the positional language model. According to the experimental results, the window-based N-gram Counting method, and Survival Analysis over an exponential distribution are the most effective among all four proposed methods, which lead to marked improvement over the baselines. This shows that the use of term proximity considerably enhances the retrieval effectiveness of the classical probabilistic models. It is therefore recommended to deploy a term proximity component in retrieval systems that employ probabilistic models.     

基于内容的图纸检索的研究与设计

基于内容图纸检索是图纸重用的关键技术,空间关系是图纸的内容的主要特征.由于空间关系描述复杂、特征向量的维数很高,在基于内容图纸检索中很难应用空间关系匹配.为了克服这些问题,提出了约简拓扑关系的方法,采用图谱方法提取拓扑特征向量,实现了特征向量的维数约减,从而通过多雏索引技术有效地加速图纸检索.综合拓扑关系、形状和方位关系作为图纸内容,有效地提高了系统查准率和查全率,通过脱机方式提取拓扑关系和形状的特征向量,而联机方式提取方位关系特征向量,提高了系统检索效率.     

Web data retrieval: solving spatial range queries using k-nearest neighbor searches

As Geographic Information Systems (GIS) technologies have evolved, more and more GIS applications and geospatial data are available on the web. Spatial objects in a given query range can be retrieved using spatial range query − one of the most widely used query types in GIS and spatial databases. However, it can be challenging to retrieve these data from various web applications where access to the data is only possible through restrictive web interfaces that support certain types of queries. A typical scenario is the existence of numerous business web sites that provide their branch locations through a limited “nearest location” web interface. For example, a chain restaurant’s web site such as McDonalds can be queried to find some of the closest locations of its branches to the user’s home address. However, even though the site has the location data of all restaurants in, for example, the state of California, it is difficult to retrieve the entire data set efficiently due to its restrictive web interface. Considering that k-Nearest Neighbor (k-NN) search is one of the most popular web interfaces in accessing spatial data on the web, this paper investigates the problem of retrieving geospatial data from the web for a given spatial range query using only k-NN searches. Based on the classification of k-NN interfaces on the web, we propose a set of range query algorithms to completely cover the rectangular shape of the query range (completeness) while minimizing the number of k-NN searches as possible (efficiency). We evaluated the efficiency of the proposed algorithms through statistical analysis and empirical experiments using both synthetic and real data sets.

Classification using proximity catch digraphs

Machine Learning - We employ random geometric digraphs to construct semi-parametric classifiers. These data-random digraphs belong to parameterized random digraph families called proximity catch...     

Efficient algorithms for spatial configuration information retrieval

The problem of spatial configuration information retrieval is a constraint satisfaction problem (CSP), which can be solved using traditional CSP algorithms. But the spatial data can be reorganized using index techniques like R-tree and the spatial data are approximated by their minimum bounding rectangles (MBRs), so the spatial configuration information retrieval is actually based on the MBRs and some special techniques can be studied. This paper studies the mapping relationships among the spatial relations for real spatial objects, the corresponding spatial relations for their MBRs and the corresponding spatial relations between the intermediate nodes and the MBRs in R-tree. Three algorithms are designed and studied, and their performances are compared.     

Knowledge-based image retrieval with spatial and temporalconstructs

A knowledge-based approach to retrieve medical images by feature and content with spatial and temporal constructs is developed. Selected objects of interest in an image are segmented and contours are generated. Features and content are extracted and stored in a database. Knowledge about image features can be expressed as a type abstraction hierarchy (TAH), the high-level nodes of which represent the most general concepts. Traversing TAH nodes allows approximate matching by feature and content if an exact match is not available. TAHs can be generated automatically by clustering algorithms based on feature values in the databases and hence are scalable to large collections of image features. Since TAHs are generated based on user classes and applications, they are context- and user-sensitive. A knowledge-based semantic image model is proposed to represent the various aspects of an image object's characteristics. The model provides a mechanism for accessing and processing spatial, evolutionary and temporal queries. A knowledge-based spatial temporal query language (KSTL) has been developed that extends ODMG's OQL and supports approximate matching of features and content, conceptual terms and temporal logic predicates. Further, a visual query language has been developed that accepts point-click-and-drag visual iconic input on the screen that is then translated into KSTL. User models are introduced to provide default parameter values for specifying query conditions. We have implemented the KMeD (Knowledge-based Medical Database) system using these concepts     

Sketch-based warping of RGBN images

While current image deformation methods are careful in making the new geometry seem right, little attention has been given to the photometric aspects. We introduce a deformation method that results in coherently illuminated objects. For this task, we use RGBN images to support a relighting step integrated in a sketch-based deformation method. We warp not only colors but also normals. Normal warping requires smooth warping fields. We use sketches to specify sparse warping samples and impose additional constraints for region of interest control. To satisfy these new constraints, we present a novel image warping method based on Hermite–Birkhoff interpolation with radial basis functions that results in a smooth warping field. We also use sketches to help the system identify both lighting conditions and material from single images. We present results with RGBN images from different sources, including photometric stereo, synthetic images, and photographs.     

Visual word proximity and linguistics for semantic video indexing and near-duplicate retrieval

Bag-of-visual-words (BoW) has recently become a popular representation to describe video and image content. Most existing approaches, nevertheless, neglect inter-word relatedness and measure similarity by bin-to-bin comparison of visual words in histograms. In this paper, we explore the linguistic and ontological aspects of visual words for video analysis. Two approaches, soft-weighting and constraint-based earth mover’s distance (CEMD), are proposed to model different aspects of visual word linguistics and proximity. In soft-weighting, visual words are cleverly weighted such that the linguistic meaning of words is taken into account for bin-to-bin histogram comparison. In CEMD, a cross-bin matching algorithm is formulated such that the ground distance measure considers the linguistic similarity of words. In particular, a BoW ontology which hierarchically specifies the hyponym relationship of words is constructed to assist the reasoning. We demonstrate soft-weighting and CEMD on two tasks: video semantic indexing and near-duplicate keyframe retrieval. Experimental results indicate that soft-weighting is superior to other popular weighting schemes such as term frequency (TF) weighting in large-scale video database. In addition, CEMD shows excellent performance compared to cosine similarity in near-duplicate retrieval.     

Sketch-based aesthetic product form exploration from existing images using piecewise clothoid curves

We present a new sketch-based product form exploration technique that works from images and sketches of existing products. At the heart of our approach, is a multi-stroke curve beautification method and a curve-based image deformation algorithm. The proposed approach converts groups of strokes into piecewise clothoid curves in order to produce visually pleasing shapes. The deformation diffusion algorithm then spatially distributes the user specified deformations through out the image to produce smooth transformations from the original image to the resulting image. We demonstrate the technique on a variety of images including photo-realistic images, real product images, and sketches.     

Video retrieval using salient foreground region of motion vector based extracted keyframes and spatial pyramid matching

Multimedia Tools and Applications - Despite enormous research efforts devoted by the research community to effectively and precisely perform video matching and retrieval among heterogeneous videos...     

Improving geographic information retrieval in spatial data infrastructures

In recent years, spatial data infrastructures (SDIs) have gained great popularity as a solution to facilitate interoperable access to geospatial data offered by different agencies. In order to enhance the data retrieval process, current infrastructures usually offer a catalog service. Nevertheless, such catalog services still have important limitations that make it difficult for users to find the geospatial data that they are interested in. Some current catalog drawbacks include the use of a single record to describe all the feature types offered by a service, the lack of formal means to describe the semantics of the underlying data, and the lack of an effective ranking metric to organize the results retrieved from a query. Aiming to overcome these limitations, this article proposes SESDI (Semantically-Enabled Spatial Data Infrastructures), which is framework that reuses techniques of classic information retrieval to improve geographic data retrieval in a SDI. Moreover, the framework proposes several ranking metrics to solve spatial, semantic, temporal and multidimensional queries.