Indexing and retrieval of images by spatial constraints

Many multimedia applications require retrieval of spatially similar images against a given query image. Existing work on image retrieval and indexing either requires extensive low-level computations or elaborate human interaction. In this paper, we introduce a new symbolic image representation technique to eliminate repetitive tasks of image understanding and object processing. Our symbolic image representation scheme is based on the concept of hierarchical decomposition of image space into spatial arrangements of features while preserving the spatial relationships among the image objects. Quadtrees are used to manage the decomposition hierarchy and play an important role in defining the similarity measure. This scheme is incremental in nature, can be adopted to accommodate varying levels of details in a wide range of application domains, and provides geometric variance independence. While ensuring that there are no false negatives, our approach also discriminates against non-matching entities by eliminating them as soon as possible, during the coarser matching phases. A hierarchical indexing scheme based on the concept of image signatures and efficient quadtree matching has been devised. Each level of the hierarchy tends to reduce the search space, allowing more involved comparisons only for potentially matching candidate database images. For a given query image, a facility is provided to rank-order the retrieved spatially similar images from the image database for subsequent browsing and selection by the user.     

Boosting image object retrieval and indexing by automatically discovered pseudo-objects

State-of-the-art object retrieval systems are mostly based on the bag-of-visual-words representation which encodes local appearance information of an image in a feature vector. An image object search is performed by comparing query object’s feature vector with those for database images. However, a database image vector generally carries mixed information of the entire image which may contain multiple objects and background. Search quality is degraded by such noisy (or diluted) feature vectors. To tackle this problem, we propose a novel representation, pseudo-objects – a subset of proximate feature points with its own feature vector to represent a local area, to approximate candidate objects in database images. In this paper, we investigate effective methods (e.g., grid, G-means, and GMM–BIC) to estimate pseudo-objects. Additionally, we also confirm that the pseudo-objects can significantly benefit inverted-file indexing both in accuracy and efficiency. Experimenting over two consumer photo benchmarks, we demonstrate that the proposed method significantly outperforms other state-of-the-art object retrieval and indexing algorithms.     

An efficient signature representation for retrieval of spatially similar images

Multimedia applications involving image retrieval demand fast and efficient response. Efficiency of search and retrieval of information in a database system is index dependent. Generally, a two-level indexing scheme in an image database can help to reduce the search space against a given query image. In such type of indexing scheme, the first level is required to significantly reduce the search space for second stage of comparisons and must be computationally efficient. It is also required to guarantee that no false negatives may result. The second level of indexing involves more detailed analysis and comparison of potentially relevant images. In this paper, we present an efficient signature representation scheme for first level of a two-level image indexing scheme that is based on hierarchical decomposition of image space into spatial arrangement of image features. Experimental results demonstrate that our signature representation scheme results in fewer number of matching signatures in the first level and significantly improves the overall computational time. As this scheme relies on corner points as the salient feature points in an image to describe its contents, we also compare results using several different contemporary corner detection methods. Further, we formally prove that the proposed signature representation scheme not only results in fewer number of signatures but also does not result in any false negative.     

Clustering of hierarchical image database to reduce inter-and intra-semantic gaps in visual space for finding specific image semantics

Empowering content based systems to assign image semantics is an interesting concept. This work explores semantically categorized image database and forms a hierarchical visual search space. Overlapping of visual features of images from different categories and subcategories are possible reasons behind inter-semantic and intra-semantic gaps. Usually each category/node in the image database has a single representation, but variability and broadness of semantic limit the usage of such representation. This work explores the application of agglomerative hierarchical clustering to automatically identify groups within a semantic in the visual space. Visual signatures of dominant clusters corresponding to a node represent its semantic. Adaptive selection of branches on this clustered data facilitates efficient semantic assignment to query image in reduced search cost. Based on the concept, content based semantic retrieval system is developed and tested on hierarchical and non-hierarchical databases. Results showcase capability of the proposed system to reduce inter- and intra-semantic gaps.     

基于潜在语义分析和相关反馈的图像检索方法

为了解决传统的CBIR系统中存在的"语义鸿沟"问题,提出一种基于潜在语义索引技术(LSI)和相关反馈技术的图像检索方法.在进行图像检索时,先在HSV空间下提取颜色直方图作为底层视觉特征进行图像检索,然后引入潜在语义索引技术试图将底层特征赋予更高层次的语义含义;并且结合相关反馈技术,通过与用户交互进一步提高检索精度.实验...     

A similarity learning approach to content-based image retrieval: application to digital mammography

In this paper, we describe an approach to content-based retrieval of medical images from a database, and provide a preliminary demonstration of our approach as applied to retrieval of digital mammograms. Content-based image retrieval (CBIR) refers to the retrieval of images from a database using information derived from the images themselves, rather than solely from accompanying text indices. In the medical-imaging context, the ultimate aim of CBIR is to provide radiologists with a diagnostic aid in the form of a display of relevant past cases, along with proven pathology and other suitable information. CBIR may also be useful as a training tool for medical students and residents. The goal of information retrieval is to recall from a database information that is relevant to the user's query. The most challenging aspect of CBIR is the definition of relevance (similarity), which is used to guide the retrieval machine. In this paper, we pursue a new approach, in which similarity is learned from training examples provided by human observers. Specifically, we explore the use of neural networks and support vector machines to predict the user's notion of similarity. Within this framework we propose using a hierarchal learning approach, which consists of a cascade of a binary classifier and a regression module to optimize retrieval effectiveness and efficiency. We also explore how to incorporate online human interaction to achieve relevance feedback in this learning framework. Our experiments are based on a database consisting of 76 mammograms, all of which contain clustered microcalcifications (MCs). Our goal is to retrieve mammogram images containing similar MC clusters to that in a query. The performance of the retrieval system is evaluated using precision-recall curves computed using a cross-validation procedure. Our experimental results demonstrate that: 1) the learning framework can accurately predict the perceptual similarity reported by human observers, thereby serving as a basis for CBIR; 2) the learning-based framework can significantly outperform a simple distance-based similarity metric; 3) the use of the hierarchical two-stage network can improve retrieval performance; and 4) relevance feedback can be effectively incorporated into this learning framework to achieve improvement in retrieval precision based on online interaction with users; and 5) the retrieved images by the network can have predicting value for the disease condition of the query.     

基于内容检索的图像数据库流水索引法的研究

图像数据库容量的增长,需要研究高效的索引技术来支持快速相似性检索的要求。总结了图像数据库检索技术的发展轨迹和特点,针对基于内容的图像检索技术中的局限性,从计算机底层硬件的角度提出了基于内容检索的流水索引法。该方法将基于内容的图像检索技术与CpU流水线结构紧密结合,对检索算法进行优化,通过举例比较,说明可提高图像数据库基于内容检索的速度。     

Generalized manifold-ranking-based image retrieval.

In this paper, we propose a general transductive learning framework named generalized manifold-ranking-based image retrieval (gMRBIR) for image retrieval. Comparing with an existing transductive learning method named MRBIR [12], our method could work well whether or not the query image is in the database; thus, it is more applicable for real applications. Given a query image, gMRBIR first initializes a pseudo seed vector based on neighborhood relationship and then spread its scores via manifold ranking to all the unlabeled images in the database. Furthermore, in gMRBIR, we also make use of relevance feedback and active learning to refine the retrieval result so that it converges to the query concept as fast as possible. Systematic experiments on a general-purpose image database consisting of 5,000 Corel images demonstrate the superiority of gMRBIR over state-of-the-art techniques.     

On Interactive Object Shape Modeling Using Algebraic Curves

Query By Sketch for indexing into an image database involves presenting the machine with a sketch of the object to be found in the database. The sketch can be of the object shape or distinct contours on the image of the object. This sketch can be made from memory, or can be refined interactively in response to what the database search returns at each iteration. Or the sketch can be made by generating curves of an object boundary or object-surface image-discontinuities from an example image. This paper introduces and describes a family of 2D curves (implicit polynomial curves) for this purpose, and an algorithm for generating a representation which passes within of a set of control points specified by the user. Control points can be placed at arbitrary locations and in arbitrary order, and can be erased by the user at will, in order to arrive at the desired shape representation. Level sets of the object potential field have been used to facilitate the interaction process. The fitting algorithm is formulated in the efficient Linear Programming (LP) framework. We illustrate the use of this method in the application of content-based image retrieval.     

Hierarchical scale-based multiobject recognition of 3-D anatomical structures

Segmentation of anatomical structures from medical images is a challenging problem, which depends on the accurate recognition (localization) of anatomical structures prior to delineation. This study generalizes anatomy segmentation problem via attacking two major challenges: 1) automatically locating anatomical structures without doing search or optimization, and 2) automatically delineating the anatomical structures based on the located model assembly. For 1), we propose intensity weighted ball-scale object extraction concept to build a hierarchical transfer function from image space to object (shape) space such that anatomical structures in 3-D medical images can be recognized without the need to perform search or optimization. For 2), we integrate the graph-cut (GC) segmentation algorithm with prior shape model. This integrated segmentation framework is evaluated on clinical 3-D images consisting of a set of 20 abdominal CT scans. In addition, we use a set of 11 foot MR images to test the generalizability of our method to the different imaging modalities as well as robustness and accuracy of the proposed methodology. Since MR image intensities do not possess a tissue specific numeric meaning, we also explore the effects of intensity nonstandardness on anatomical object recognition. Experimental results indicate that: 1) effective recognition can make the delineation more accurate; 2) incorporating a large number of anatomical structures via a model assembly in the shape model improves the recognition and delineation accuracy dramatically; 3) ball-scale yields useful information about the relationship between the objects and the image; 4) intensity variation among scenes in an ensemble degrades object recognition performance.     

Design and analysis of a content-based pathology image retrieval system

A prototype, content-based image retrieval system has been built employing a client/server architecture to access supercomputing power from the physician's desktop. The system retrieves images and their associated annotations from a networked microscopic pathology image database based on content similarity to user supplied query images. Similarity is evaluated based on four image feature types: color histogram, image texture, Fourier coefficients, and wavelet coefficients, using the vector dot product as a distance metric. Current retrieval accuracy varies across pathological categories depending on the number of available training samples and the effectiveness of the feature set. The distance measure of the search algorithm was validated by agglomerative cluster analysis in light of the medical domain knowledge. Results show a correlation between pathological significance and the image document distance value generated by the computer algorithm. This correlation agrees with observed visual similarity. This validation method has an advantage over traditional statistical evaluation methods when sample size is small and where domain knowledge is important. A multi-dimensional scaling analysis shows a low dimensionality nature of the embedded space for the current test set.     

文本分类检索技术在工程中的应用

根据工程项目中文本特点与用户业务需求,基于向量空间模型,结合示例检索与分类检索技术,设计与实现了文本分类检索系统,以面向对象中类图模型形式描述了系统的实现方法。为解决工程应用中出现的性能等问题,提出了各种改进优化方法,如采用特征提取将向量降维,减少存储空间,提高计算速度;采用分类检索缩小搜索范围,缩小检索时间;定时增量更新向量空间提高处理速度,最终使系统成功应用到了工程项目中。     

Semantic clustering for region-based image retrieval

With the proliferation of applications that demand content-based image retrieval, two merits are becoming more desirable. The first is the reduced search space, and the second is the reduced “semantic gap.” This paper proposes a semantic clustering scheme to achieve these two goals. By performing clustering before image retrieval, the search space can be significantly reduced. The proposed method is different from existing image clustering methods as follows: (1) it is region based, meaning that image sub-regions, instead of the whole image, are grouped into. The semantic similarities among image regions are collected over the user query and feedback history; (2) the clustering scheme is dynamic in the sense that it can evolve to include more new semantic categories. Ideally, one cluster approximates one semantic concept or a small set of closely related semantic concepts, based on which the “semantic gap” in the retrieval is reduced.     

Effective image retrieval based on hidden concept discovery in image database.

This paper addresses content-based image retrieval in general, and in particular, focuses on developing a hidden semantic concept discovery methodology to address effective semantics-intensive image retrieval. In our approach, each image in the database is segmented into regions associated with homogenous color, texture, and shape features. By exploiting regional statistical information in each image and employing a vector quantization method, a uniform and sparse region-based representation is achieved. With this representation, a probabilistic model based on statistical-hidden-class assumptions of the image database is obtained, to which the expectation-maximization technique is applied to analyze semantic concepts hidden in the database. An elaborated retrieval algorithm is designed to support the probabilistic model. The semantic similarity is measured through integrating the posterior probabilities of the transformed query image, as well as a constructed negative example, to the discovered semantic concepts. The proposed approach has a solid statistical foundation; the experimental evaluations on a database of 10000 general-purposed images demonstrate its promise and effectiveness.     

基于多特征的对象检索技术

提出了一种融合图像颜色、纹理和形状特征的提取及归一化方法,并将其应用于基于内容的对象检索中,实验证明,融合颜色、纹理、和形状特征的对象检索比单一特征的对象检索效果要好。     

A unified image retrieval framework on local visual and semantic concept-based feature spaces

This paper presents a learning-based unified image retrieval framework to represent images in local visual and semantic concept-based feature spaces. In this framework, a visual concept vocabulary (codebook) is automatically constructed by utilizing self-organizing map (SOM) and statistical models are built for local semantic concepts using probabilistic multi-class support vector machine (SVM). Based on these constructions, the images are represented in correlation and spatial relationship-enhanced concept feature spaces by exploiting the topology preserving local neighborhood structure of the codebook, local concept correlation statistics, and spatial relationships in individual encoded images. Finally, the features are unified by a dynamically weighted linear combination of similarity matching scheme based on the relevance feedback information. The feature weights are calculated by considering both the precision and the rank order information of the top retrieved relevant images of each representation, which adapts itself to individual searches to produce effective results. The experimental results on a photographic database of natural scenes and a bio-medical database of different imaging modalities and body parts demonstrate the effectiveness of the proposed framework.