Silhouette-based occluded object recognition through curvature scale space

A complete and practical system for occluded object recognition has been developed which is very robust with respect to noise and local deformations of shape (due to weak perspective distortion, segmentation errors and non-rigid material) as well as scale, position and orientation changes of the objects. The system has been tested on a wide variety of free-form 3D objects. An industrial application is envisaged where a fixed camera and a light-box are utilized to obtain images. Within the constraints of the system, every rigid 3D object can be modeled by a limited number of classes of 2D contours corresponding to the object's resting positions on the light-box. The contours in each class are related to each other by a 2D similarity transformation. The Curvature Scale Space technique [26, 28] is then used to obtain a novel multi-scale segmentation of the image and the model contours. Object indexing [16, 32, 36] is used to narrow down the search space. An efficient local matching algorithm is utilized to select the best matching models. Received: 5 August 1996 / Accepted: 19 March 1997     

Affine Curvature Scale Space with Affine Length Parametrisation

The maxima of Curvature Scale Space (CSS) image have been used to represent 2D shapes under affine transforms. The CSS image is expected to be in the MPEG-7 package of standards. Since the CSS image employs the arc length parametrisation which is not affine invariant, we expect some deviations in the maxima of the CSS image under general affine transforms. Affine length and affine curvature have already been introduced and used as alternatives to arc length and conventional curvature in affine transformed environments. The utility of using these parameters to enrich the CSS representation is addressed in this paper. We use arc length to parametrise the curve prior to computing its CSS image. The parametrisation has been proven to be invariant under affine transformation and has been used in many affine invariant shape recognition methods. Since the organisation of the CSS image is based on curvature zero crossings of the curve, in this paper, we also investigate the advantages and shortcomings of using affine curvature in computation of the CSS image. The enriched CSS representations are then used to find similar shapes from a very large prototype database, and also a small classified database, both consisting of original as well as affine transformed shapes. An improvement is observed over the conventional CSS image.     

Using extended feature objects for partial similarity retrieval

In this paper, we introduce the concept of extended feature objects for similarity retrieval. Conventional approaches for similarity search in databases map each object in the database to a point in some high-dimensional feature space and define similarity as some distance measure in this space. For many similarity search problems, this feature-based approach is not sufficient. When retrieving partially similar polygons, for example, the search cannot be restricted to edge sequences, since similar polygon sections may start and end anywhere on the edges of the polygons. In general, inherently continuous problems such as the partial similarity search cannot be solved by using point objects in feature space. In our solution, we therefore introduce extended feature objects consisting of an infinite set of feature points. For an efficient storage and retrieval of the extended feature objects, we determine the minimal bounding boxes of the feature objects in multidimensional space and store these boxes using a spatial access structure. In our concrete polygon problem, sets of polygon sections are mapped to 2D feature objects in high-dimensional space which are then approximated by minimal bounding boxes and stored in an R-tree. The selectivity of the index is improved by using an adaptive decomposition of very large feature objects and a dynamic joining of small feature objects. For the polygon problem, translation, rotation, and scaling invariance is achieved by using the Fourier-transformed curvature of the normalized polygon sections. In contrast to vertex-based algorithms, our algorithm guarantees that no false dismissals may occur and additionally provides fast search times for realistic database sizes. We evaluate our method using real polygon data of a supplier for the car manufacturing industry. Edited by R. Güting. Received October 7, 1996 / Accepted March 28, 1997     

Approximate similarity retrieval with M-trees

Motivated by the urgent need to improve the efficiency of similarity queries, approximate similarity retrieval is investigated in the environment of a metric tree index called the M-tree. Three different approximation techniques are proposed, which show how to forsake query precision for improved performance. Measures are defined that can quantify the improvements in performance efficiency and the quality of approximations. The proposed approximation techniques are then tested on various synthetic and real-life files. The evidence obtained from the experiments confirms our hypothesis that a high-quality approximated similarity search can be performed at a much lower cost than that needed to obtain the exact results. The proposed approximation techniques are scalable and appear to be independent of the metric used. Extensions of these techniques to the environments of other similarity search indexes are also discussed. Received July 7, 1998 / Accepted October 13, 1998     

Shape similarity by homotopic deformation

nk +n log n) time, where n is the number of vertices in the polygonal shapes and k is an index to indicate how the target shape is convoluted. Experimental results show the feasibility of our approach. Extending the method for comparison of scenes with mutiple objects is also discussed.     

Discovery of a perceptual distance function for measuring image similarity

For more than a decade, researchers have actively explored the area of image/video analysis and retrieval. Yet one fundamental problem remains largely unsolved: how to measure perceptual similarity between two objects. For this purpose, most researchers employ a Minkowski-type metric. Unfortunately, the Minkowski metric does not reliably find similarities in objects that are obviously alike. Through mining a large set of visual data, our team has discovered a perceptual distance function. We call the discovered function the dynamic partial function (DPF). When we empirically compare DPF to Minkowski-type distance functions in image retrieval and in video shot-transition detection using our image features, DPF performs significantly better. The effectiveness of DPF can be explained by similarity theories in cognitive psychology.     

A new image registration scheme based on curvature scale space curve matching

We propose a new image registration scheme for remote sensing images. This scheme includes three steps in sequence. First, a segmentation process is performed on the input image pair. Then the boundaries of the segmented regions in two images are extracted and matched. These matched regions are called confidence regions. Finally, a non-linear optimization is performed in the matched regions only to obtain a global set of transform parameters. Experiments show that this scheme is more robust and converges faster than registration of the original image pair. We also develop a new curve-matching algorithm based on curvature scale space to facilitate the second step.     

Image retrieval by color semantics

The development of a system supporting querying of image databases by color content tackles a major design choice about properties of colors which are referenced within user queries. On the one hand, low-level properties directly reflect numerical features and concepts tied to the machine representation of color information. On the other hand, high-level properties address concepts such as the perceptual quality of colors and the sensations that they convey. Color-induced sensations include warmth, accordance or contrast, harmony, excitement, depression, anguish, etc. In other words, they refer to the semantics of color usage. In particular, paintings are an example where the message is contained more in the high-level color qualities and spatial arrangements than in the physical properties of colors. Starting from this observation, Johannes Itten introduced a formalism to analyze the use of color in art and the effects that this induces on the user's psyche. In this paper, we present a system which translates the Itten theory into a formal language that expresses the semantics associated with the combination of chromatic properties of color images. The system exploits a competitive learning technique to segment images into regions with homogeneous colors. Fuzzy sets are used to represent low-level region properties such as hue, saturation, luminance, warmth, size and position. A formal language and a set of model-checking rules are implemented to define semantic clauses and verify the degree of truth by which they hold over an image.     

Fast image retrieval using color-spatial information

In this paper, we present an image retrieval system that employs both the color and spatial information of images to facilitate the retrieval process. The basic unit used in our technique is a single-colored cluster, which bounds a homogeneous region of that color in an image. Two clusters from two images are similar if they are of the same color and overlap in the image space. The number of clusters that can be extracted from an image can be very large, and it affects the accuracy of retrieval. We study the effect of the number of clusters on retrieval effectiveness to determine an appropriate value for “optimal' performance. To facilitate efficient retrieval, we also propose a multi-tier indexing mechanism called the Sequenced Multi-Attribute Tree (SMAT). We implemented a two-tier SMAT, where the first layer is used to prune away clusters that are of different colors, while the second layer discriminates clusters of different spatial locality. We conducted an experimental study on an image database consisting of 12,000 images. Our results show the effectiveness of the proposed color-spatial approach, and the efficiency of the proposed indexing mechanism. Received August 1, 1997 / Accepted December 9, 1997     

Relevance feedback in image retrieval: A comprehensive review

We analyze the nature of the relevance feedback problem in a continuous representation space in the context of content-based image retrieval. Emphasis is put on exploring the uniqueness of the problem and comparing the assumptions, implementations, and merits of various solutions in the literature. An attempt is made to compile a list of critical issues to consider when designing a relevance feedback algorithm. With a comprehensive review as the main portion, this paper also offers some novel solutions and perspectives throughout the discussion. RID="*" ID="*" Work was done while at the University of Illinois.     

Image retrieval by texture similarity

Accuracy and efficiency are the two important issues in designing content-based image retrieval systems. In this paper, we present an efficient image retrieval system with high performance of accuracy based on two novel features, the composite sub-band gradient vector and the energy distribution pattern string. Both features are generated from the sub-images of a wavelet decomposition of the original image. A fuzzy matching mechanism based on energy distribution pattern strings serves as a filter to quickly remove undesired images in the database from further consideration. The images passing the filter will be compared with the query image based on composite sub-band gradient vectors which are extremely powerful for discriminating detailed textures. Through several extensive experiments by exercising our prototype system with a database of 2400 images, we demonstrated that both high accuracy and high efficiency can be achieved at the same time by our approach.     

Deformation similarity measurement in quasi-conformal shape space

This paper presents a novel approach based on the shape space concept to classify deformations of 3D models. A new quasi-conformal metric is introduced which measures the curvature changes at each vertex of each pose during the deformation. The shapes with similar deformation patterns follow a similar deformation curve in shape space. Energy functional of the deformation curve is minimized to calculate the geodesic curve connecting two shapes on the shape space manifold. The geodesic distance illustrates the similarity between two shapes, which is used to compute the similarity between the deformations. We applied our method to classify the left ventricle deformations of myopathic and control subjects, and the sensitivity and specificity of our method were 88.8% and 85.7%, which are higher than other methods based on the left ventricle cavity, which shows our method can quantify the similarity and disparity of the left ventricle motion well.     

NeTra: A toolbox for navigating large image databases

We present here an implementation of NeTra, a prototype image retrieval system that uses color, texture, shape and spatial location information in segmented image regions to search and retrieve similar regions from the database. A distinguishing aspect of this system is its incorporation of a robust automated image segmentation algorithm that allows object- or region-based search. Image segmentation significantly improves the quality of image retrieval when images contain multiple complex objects. Images are segmented into homogeneous regions at the time of ingest into the database, and image attributes that represent each of these regions are computed. In addition to image segmentation, other important components of the system include an efficient color representation, and indexing of color, texture, and shape features for fast search and retrieval. This representation allows the user to compose interesting queries such as “retrieve all images that contain regions that have the color of object A, texture of object B, shape of object C, and lie in the upper of the image”, where the individual objects could be regions belonging to different images. A Java-based web implementation of NeTra is available at http://vivaldi.ece.ucsb.edu/Netra.     

Automatic extraction of roads from aerial images based on scale space and snakes

Abstract. We propose a new approach for automatic road extraction from aerial imagery with a model and a strategy mainly based on the multi-scale detection of roads in combination with geometry-constrained edge extraction using snakes. A main advantage of our approach is, that it allows for the first time a bridging of shadows and partially occluded areas using the heavily disturbed evidence in the image. Additionally, it has only few parameters to be adjusted. The road network is constructed after extracting crossings with varying shape and topology. We show the feasibility of the approach not only by presenting reasonable results but also by evaluating them quantitatively based on ground truth. Received: 22 July 1999 / Accepted: 20 March 2000