Multi-object image retrieval based on shape and topology

We aim at developing a geometry-based retrieval system for multi-object images. We model both shape and topology of image objects including holes using a structured representation called curvature tree (CT); the hierarchy of the CT reflects the inclusion relationships between the objects and holes. To facilitate shape-based matching, triangle-area representation (TAR) of each object and hole is stored at the corresponding node in the CT. The similarity between two multi-object images is measured based on the maximum similarity subtree isomorphism (MSSI) between their CTs. For this purpose, we adapt a continuous optimization approach to solve the MSSI problem and a very effective dynamic programming algorithm to measure the similarity between the attributed nodes. Our matching scheme agrees with many recent findings in psychology about the human perception of multi-object images. Experiments on a database of 1500 logos and the MPEG-7 CE-1 database of 1400 shape images have shown the significance of the proposed method.     

Indexing and retrieval of images by spatial constraints

Many multimedia applications require retrieval of spatially similar images against a given query image. Existing work on image retrieval and indexing either requires extensive low-level computations or elaborate human interaction. In this paper, we introduce a new symbolic image representation technique to eliminate repetitive tasks of image understanding and object processing. Our symbolic image representation scheme is based on the concept of hierarchical decomposition of image space into spatial arrangements of features while preserving the spatial relationships among the image objects. Quadtrees are used to manage the decomposition hierarchy and play an important role in defining the similarity measure. This scheme is incremental in nature, can be adopted to accommodate varying levels of details in a wide range of application domains, and provides geometric variance independence. While ensuring that there are no false negatives, our approach also discriminates against non-matching entities by eliminating them as soon as possible, during the coarser matching phases. A hierarchical indexing scheme based on the concept of image signatures and efficient quadtree matching has been devised. Each level of the hierarchy tends to reduce the search space, allowing more involved comparisons only for potentially matching candidate database images. For a given query image, a facility is provided to rank-order the retrieved spatially similar images from the image database for subsequent browsing and selection by the user.     

Similarity measure for superquadrics

Superquadrics with parametric deformations are suitable models for use as solid primitives for describing a complicated 3-D object. Some different methods for the recovery of superquadric primitives from range data have been proposed, but there is still no effective similarity measure for the matching task between two superquadrics in a 3-D object recognition system. The authors propose a similarity measure to evaluate the degree of shape similarity between two superquadric-based objects. This similarity measure is defined as the volume of regions bounded by the surfaces of two 3-D objects. The proposed measure has been proved to be a metric. The metric value is computed by the Monte Carlo integration method. The experimental results illustrate that the proposed similarity measure is effective in matching a recovered superquadric with a set of superquadrics in the model database     

3-d modeling from a single view of a symmetric object

3-D technologies are considered as the next generation of multimedia applications. Currently, one of the challenges faced by 3-D applications is the shortage of 3-D resources. To solve this problem, many 3-D modeling methods are proposed to directly recover 3-D geometry from 2-D images. However, these methods on single view modeling either require intensive user interaction, or are restricted to a specific kind of object. In this paper, we propose a novel 3-D modeling approach to recover 3-D geometry from a single image of a symmetric object with minimal user interaction. Symmetry is one of the most common properties of natural or manmade objects. Given a single view of a symmetric object, the user marks some symmetric lines and depth discontinuity regions on the image. Our algorithm first finds a set of planes to approximately fit to the object, and then a rough 3-D point cloud is generated by an optimization procedure. The occluded part of the object is further recovered using symmetry information. Experimental results on various indoor and outdoor objects show that the proposed system can obtain 3-D models from single images with only a little user interaction.     

Learning bayesian classifiers for scene classification with a visual grammar

A challenging problem in image content extraction and classification is building a system that automatically learns high-level semantic interpretations of images. We describe a Bayesian framework for a visual grammar that aims to reduce the gap between low-level features and high-level user semantics. Our approach includes modeling image pixels using automatic fusion of their spectral, textural, and other ancillary attributes; segmentation of image regions using an iterative split-and-merge algorithm; and representing scenes by decomposing them into prototype regions and modeling the interactions between these regions in terms of their spatial relationships. Naive Bayes classifiers are used in the learning of models for region segmentation and classification using positive and negative examples for user-defined semantic land cover labels. The system also automatically learns representative region groups that can distinguish different scenes and builds visual grammar models. Experiments using Landsat scenes show that the visual grammar enables creation of high-level classes that cannot be modeled by individual pixels or regions. Furthermore, learning of the classifiers requires only a few training examples.     

On Interactive Object Shape Modeling Using Algebraic Curves

Query By Sketch for indexing into an image database involves presenting the machine with a sketch of the object to be found in the database. The sketch can be of the object shape or distinct contours on the image of the object. This sketch can be made from memory, or can be refined interactively in response to what the database search returns at each iteration. Or the sketch can be made by generating curves of an object boundary or object-surface image-discontinuities from an example image. This paper introduces and describes a family of 2D curves (implicit polynomial curves) for this purpose, and an algorithm for generating a representation which passes within of a set of control points specified by the user. Control points can be placed at arbitrary locations and in arbitrary order, and can be erased by the user at will, in order to arrive at the desired shape representation. Level sets of the object potential field have been used to facilitate the interaction process. The fitting algorithm is formulated in the efficient Linear Programming (LP) framework. We illustrate the use of this method in the application of content-based image retrieval.     

Background removal of multiview images by learning shape priors.

Image-based rendering has been successfully used to display 3-D objects for many applications. A well-known example is the object movie, which is an image-based 3-D object composed of a collection of 2-D images taken from many different viewpoints of a 3-D object. In order to integrate image-based 3-D objects into a chosen scene (e.g., a panorama), one has to meet a hard challenge--to efficiently and effectively remove the background from the foreground object. This problem is referred to as multiview images (MVIs) segmentation. Another task requires MVI segmentation is image-based 3-D reconstruction using multiview images. In this paper, we propose a new method for segmenting MVI, which integrates some useful algorithms, including the well-known graph-cut image segmentation and volumetric graph-cut. The main idea is to incorporate the shape prior into the image segmentation process. The shape prior introduced into every image of the MVI is extracted from the 3-D model reconstructed by using the volumetric graph cuts algorithm. Here, the constraint obtained from the discrete medial axis is adopted to improve the reconstruction algorithm. The proposed MVI segmentation process requires only a small amount of user intervention, which is to select a subset of acceptable segmentations of the MVI after the initial segmentation process. According to our experiments, the proposed method can provide not only good MVI segmentation, but also provide acceptable 3-D reconstructed models for certain less-demanding applications.     

Object-based analysis-synthesis coding (OBASC) based on the sourcemodel of moving flexible 3-D objects

This correspondence investigates object-based analysis-synthesis coding (OBASC) for the encoding of moving images at very low data rates. According to the source model, each moving object of an image is described and encoded by three parameter sets defining its motion, shape, and surface color. The parameter sets of each object are obtained by model-based image analysis. They are coded by an object-dependent parameter coding. Using the coded parameter sets, an image can be synthesized by model-based image synthesis. Here, OBASC based on the source model of "moving flexible 3-D objects with 3-D motion" (F3D) is introduced. The efficiency of this source model F3D is compared to the efficiency of OBASC based on the source model of "moving rigid 3-D objects with 3-D motion" (R3D). Compared to R3D, F3D requires the additional transmission of flexible-shape parameters. Therefore, the source model F3D is only applied in those areas of the image which cannot be described by the source model R3D. The new source model F3D reduces the bit rate from 64 to 56 kb/s, providing the same picture quality measured by the SNR of the encoded color parameters.     

Preferential Image Segmentation Using Trees of Shapes

A novel preferential image segmentation method is proposed that performs image segmentation and object recognition using mathematical morphologies. The method preferentially segments objects that have intensities and boundaries similar to those of objects in a database of prior images. A tree of shapes is utilized to represent the content distributions in images, and curve matching is applied to compare the boundaries. The algorithm is invariant to contrast change and similarity transformations of translation, rotation and scale. A performance evaluation of the proposed method using a large image dataset is provided. Experimental results show that the proposed approach is promising for applications such as object segmentation and video tracking with cluttered backgrounds.      

An effective use of adaptive combination of visual features to retrieve image semantics from a hierarchical image database

Correlating semantic and visual similarity of an image is a challenging task. Unlimited possibilities of objects classification in real world are challenges for learning based techniques. Semantics based categorization of images gives a semantically categorized hierarchical image database. This work utilizes the strength of such database and proposes a system for automatic semantics assignment to images using an adaptive combination of multiple visual features. ‘Branch Selection Algorithm’ selects only a few subtrees to search from this image database. Pruning Algorithms further reduce this search space. Correlation of semantic and visual similarities is also explored to understand overlapping of semantics in visual space. The efficacy of the proposed algorithms analyzed on hierarchical and non-hierarchical databases shows that the system is capable of assigning accurate general and specific semantics to images automatically.     

Recursive Representation and Progressive Display of Binary Objects for Efficient Network Browsing

When binary objects are browsed in a network environment, data transmission rate, progressive display capability, and view modification under rotation, scaling, and/or translation (R/S/T) are the major factors for selection of an appropriate representation model of binary objects. A new half-plane-based representation and display method for 2D binary objects is proposed. Within this modeling framework, a binary object approximated by a shape of a polygon can be represented as a collection of half-planes defined over the edges of the polygon under operations of union and intersection. The basic shape attributes of the object model are the slope and they-intercept of the boundary line of the constituent half planes. In the progressive display of the binary object our method adopts the quadtree block subdivision to divide the object image into hierarchical levels of detail (or resolution). Our method determines the color of a quadtree node based on the (angle, intercept) representation parameters. It is shown that the representation parameters at the parent node are recursively related to those at the child nodes. This recursive relation is crucial for deriving the color of the nodes for progressive object display. Lemmas for the node color determination for an object expressed in the form of half-planes, a convex polygon, or a concave polygon are derived step by step. Our method is generally better than many existing methods in terms of data transmission rate, progressive display capability, and view modification under R/S/T variations. Simulation results are provided to illustrate the performance of our method.     

PicToSeek: combining color and shape invariant features for imageretrieval

We aim at combining color and shape invariants for indexing and retrieving images. To this end, color models are proposed independent of the object geometry, object pose, and illumination. From these color models, color invariant edges are derived from which shape invariant features are computed. Computational methods are described to combine the color and shape invariants into a unified high-dimensional invariant feature set for discriminatory object retrieval. Experiments have been conducted on a database consisting of 500 images taken from multicolored man-made objects in real world scenes. From the theoretical and experimental results it is concluded that object retrieval based on composite color and shape invariant features provides excellent retrieval accuracy. Object retrieval based on color invariants provides very high retrieval accuracy whereas object retrieval based entirely on shape invariants yields poor discriminative power. Furthermore, the image retrieval scheme is highly robust to partial occlusion, object clutter and a change in the object's pose. Finally, the image retrieval scheme is integrated into the PicToSeek system on-line at http://www.wins.uva.nl/research/isis/PicToSeek/ for searching images on the World Wide Web.     

Brain anatomical structure segmentation by hybrid discriminative/generative models

In this paper, a hybrid discriminative/generative model for brain anatomical structure segmentation is proposed. The learning aspect of the approach is emphasized. In the discriminative appearance models, various cues such as intensity and curvatures are combined to locally capture the complex appearances of different anatomical structures. A probabilistic boosting tree (PBT) framework is adopted to learn multiclass discriminative models that combine hundreds of features across different scales. On the generative model side, both global and local shape models are used to capture the shape information about each anatomical structure. The parameters to combine the discriminative appearance and generative shape models are also automatically learned. Thus, low-level and high-level information is learned and integrated in a hybrid model. Segmentations are obtained by minimizing an energy function associated with the proposed hybrid model. Finally, a grid-face structure is designed to explicitly represent the 3-D region topology. This representation handles an arbitrary number of regions and facilitates fast surface evolution. Our system was trained and tested on a set of 3-D magnetic resonance imaging (MRI) volumes and the results obtained are encouraging.     

Hierarchical scale-based multiobject recognition of 3-D anatomical structures

Segmentation of anatomical structures from medical images is a challenging problem, which depends on the accurate recognition (localization) of anatomical structures prior to delineation. This study generalizes anatomy segmentation problem via attacking two major challenges: 1) automatically locating anatomical structures without doing search or optimization, and 2) automatically delineating the anatomical structures based on the located model assembly. For 1), we propose intensity weighted ball-scale object extraction concept to build a hierarchical transfer function from image space to object (shape) space such that anatomical structures in 3-D medical images can be recognized without the need to perform search or optimization. For 2), we integrate the graph-cut (GC) segmentation algorithm with prior shape model. This integrated segmentation framework is evaluated on clinical 3-D images consisting of a set of 20 abdominal CT scans. In addition, we use a set of 11 foot MR images to test the generalizability of our method to the different imaging modalities as well as robustness and accuracy of the proposed methodology. Since MR image intensities do not possess a tissue specific numeric meaning, we also explore the effects of intensity nonstandardness on anatomical object recognition. Experimental results indicate that: 1) effective recognition can make the delineation more accurate; 2) incorporating a large number of anatomical structures via a model assembly in the shape model improves the recognition and delineation accuracy dramatically; 3) ball-scale yields useful information about the relationship between the objects and the image; 4) intensity variation among scenes in an ensemble degrades object recognition performance.     

Mining user hidden semantics from image content for image retrieval

The problem confronted in the content-based image retrieval research is the semantic gap between the low-level feature representing and high-level semantics in the images. This paper describes a way to bridge such gap: by learning the similar images given from the user, the system extracts the similar region pairs and classifies those similar region pairs either as object or non-object semantics, and either as object-relation or non-object-relation semantics automatically, which are obtained from comparing the distances and spatial relationships in the similar region pairs by themselves. The system also extracts interesting parts of the features from the similar region pair and then adjusts each interesting feature and region pair weight dynamically. Using those objects and object-relation semantics as well as the dynamic weights adjustment from the similar images, the semantics of those similar images can be mined and used for searching the similar images. The experiments show that the proposed system can retrieve the similar images well and efficient.     

基于多特征扩散方法的显著性物体检测

现有的大部分基于扩散理论的显著性物体检测方法只用了图像的底层特征来构造图和扩散矩阵,并且忽视了显著性物体在图像边缘的可能性。针对此,该文提出一种基于图像的多层特征的扩散方法进行显著性物体检测。首先,采用由背景先验、颜色先验、位置先验组成的高层先验方法选取种子节点。其次,将选取的种子节点的显著性信息通过由图像的底层特征构建的扩散矩阵传播到每个节点得到初始显著图,并将其作为图像的中层特征。然后结合图像的高层特征分别构建扩散矩阵,再次运用扩散方法分别获得中层显著图、高层显著图。最后,非线性融合中层显著图和高层显著图得到最终显著图。该算法在3个数据集MSRA10K,DUT-OMRON和ECSSD上,用3种量化评价指标与现有4种流行算法进行实验结果对比,均取得最好的效果。