Strategies of Multi-view and Multi-matching for 3D Object Recognition

A multi-view representation scheme and a multi-matching strategy for 3D object recognition are described; 3D objects are represented in terms of their 2D appearances so that 2D techniques can be applied to 3D recognition. Appearances of objects in the representation scheme are further organized in a hierarchical manner so that the matching process can reduce its search space by examining only the optimal view at every level of the representation scheme. In our multi-matching strategy, the matching module is composed of four components: point matcher, string matcher, vector matcher, and chamfer matcher. Each matcher is associated with a termination rule so that impossible views can be rejected at the early stages of the matching process. Experimental results reveal that the proposed strategies are feasible for 3D object recognition.     

Visual learning and recognition of 3-d objects from appearance

The problem of automatically learning object models for recognition and pose estimation is addressed. In contrast to the traditional approach, the recognition problem is formulated as one of matching appearance rather than shape. The appearance of an object in a two-dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties and constant for a rigid object, pose and illumination vary from scene to scene. A compact representation of object appearance is proposed that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a manifold. Given an unknown input image, the recognition system projects the image to eigenspace. The object is recognized based on the manifold it lies on. The exact position of the projection on the manifold determines the object's pose in the image.A variety of experiments are conducted using objects with complex appearance characteristics. The performance of the recognition and pose estimation algorithms is studied using over a thousand input images of sample objects. Sensitivity of recognition to the number of eigenspace dimensions and the number of learning samples is analyzed. For the objects used, appearance representation in eigenspaces with less than 20 dimensions produces accurate recognition results with an average pose estimation error of about 1.0 degree. A near real-time recognition system with 20 complex objects in the database has been developed. The paper is concluded with a discussion on various issues related to the proposed learning and recognition methodology.     

Sparse representation for coarse and fine object recognition

This paper offers a sparse, multiscale representation of objects. It captures the object appearance by selection from a very large dictionary of Gaussian differential basis functions. The learning procedure results from the matching pursuit algorithm, while the recognition is based on polynomial approximation to the bases, turning image matching into a problem of polynomial evaluation. The method is suited for coarse recognition between objects and, by adding more bases, also for fine recognition of the object pose. The advantages over the common representation using PCA include storing sampled points for recognition is not required, adding new objects to an existing data set is trivial because retraining other object models is not needed, and significantly in the important case where one has to scan an image over multiple locations in search for an object, the new representation is readily available as opposed to PCA projection at each location. The experimental result on the COIL-100 data set demonstrates high recognition accuracy with real-time performance.     

A neural network approach to CSG-based 3-D object recognition

Describes the recognition subsystem of a computer vision system based on constructive solid geometry (CSG) representation scheme. Instead of using the conventional CSG trees to represent objects, the proposed system uses an equivalent representation scheme-precedence graphs-for object representation. Each node in the graph represents a primitive volume and each are between two nodes represents the relation between them. Object recognition is achieved by matching the scene precedence graph to the model precedence graph. A constraint satisfaction network is proposed to implement the matching process. The energy function associated with the network is used to enforce the matching constraints including match validity, primitive similarity, precedence graph preservation, and geometric structure preservation. The energy level is at its minimum only when the optimal match is reached. Experimental results on several range images are presented to demonstrate the proposed approach     

Scalable representation for 3D object recognition using feature sharing and view clustering

Scalability is an important issue in object recognition as it reduces database storage and recognition time. In this paper, we propose a new scalable 3D object representation and a learning method to recognize many everyday objects. The key proposal for scalable object representation is to combine the concept of feature sharing with multi-view clustering in part-based object representation, in particular a common-frame constellation model (CFCM). In this representation scheme, we also propose a fully automatic learning method: appearance-based automatic feature clustering and sequential construction of clustered CFCMs from labeled multi-views and multiple objects. We evaluated the scalability of the proposed method to COIL-100 DB and applied the learning scheme to 112 objects with 620 training views. Experimental results show the scalable learning results in almost constant recognition performance relative to the number of objects.     

Silhouette-based isolated object recognition through curvaturescale space

A complete, fast and practical isolated object recognition system has been developed which is very robust with respect to scale, position and orientation changes of the objects as well as noise and local deformations of shape (due to perspective projection, segmentation errors and non-rigid material used in some objects). The system has been tested on a wide variety of three-dimensional objects with different shapes and material and surface properties. A light-box setup is used to obtain silhouette images which are segmented to obtain the physical boundaries of the objects which are classified as either convex or concave. Convex curves are recognized using their four high-scale curvature extrema points. Curvature scale space (CSS) representations are computed for concave curves. The CSS representation is a multi-scale organization of the natural, invariant features of a curve (curvature zero-crossings or extrema) and useful for very reliable recognition of the correct model since it places no constraints on the shape of objects. A three-stage, coarse-to-fine matching algorithm prunes the search space in stage one by applying the CSS aspect ratio test. The maxima of contours in CSS representations of the surviving models are used for fast CSS matching in stage two. Finally, stage three verifies the best match and resolves any ambiguities by determining the distance between the image and model curves. Transformation parameter optimization is then used to find the best fit of the input object to the correct model     

Recognition of shapes by attributed skeletal graphs

In this paper, we propose a framework to address the problem of generic 2-D shape recognition. The aim is mainly on using the potential strength of skeleton of discrete objects in computer vision and pattern recognition where features of objects are needed for classification. We propose to represent the medial axis characteristic points as an attributed skeletal graph to model the shape. The information about the object shape and its topology is totally embedded in them and this allows the comparison of different objects by graph matching algorithms. The experimental results demonstrate the correctness in detecting its characteristic points and in computing a more regular and effective representation for a perceptual indexing. The matching process, based on a revised graduated assignment algorithm, has produced encouraging results, showing the potential of the developed method in a variety of computer vision and pattern recognition domains. The results demonstrate its robustness in the presence of scale, reflection and rotation transformations and prove the ability to handle noise and occlusions.     

Efficient data structures for model-based 3-D object recognitionand localization from range images

An effective method of surface characterization of 3D objects using surface curvature properties and an efficient approach to recognizing and localizing multiple 3D free-form objects (free-form object recognition and localization) are presented. The approach is surface based and is therefore not sensitive to noise and occlusion, forms hypothesis by local analysis of surface shapes, does not depend on the visibility of complete objects, and uses information from a CAD database in recognition and localization. A knowledge representation scheme for describing free-form surfaces is described. The data structure and procedures are well designed, so that the knowledge leads the system to intelligent behavior. Knowledge about surface shapes is abstracted from CAD models to direct the search in verification of vision hypotheses. The knowledge representation used eases processes of knowledge acquisition, information retrieval, modification of knowledge base, and reasoning for solution     

Accurate Object Recognition with Shape Masks

In this paper we propose an object recognition approach that is based on shape masks—generalizations of segmentation masks. As shape masks carry information about the extent (outline) of objects, they provide a convenient tool to exploit the geometry of objects. We apply our ideas to two common object class recognition tasks—classification and localization. For classification, we extend the orderless bag-of-features image representation. In the proposed setup shape masks can be seen as weak geometrical constraints over bag-of-features. Those constraints can be used to reduce background clutter and help recognition. For localization, we propose a new recognition scheme based on high-dimensional hypothesis clustering. Shape masks allow to go beyond bounding boxes and determine the outline (approximate segmentation) of the object during localization. Furthermore, the method easily learns and detects possible object viewpoints and articulations, which are often well characterized by the object outline. Our experiments reveal that shape masks can improve recognition accuracy of state-of-the-art methods while returning richer recognition answers at the same time. We evaluate the proposed approach on the challenging natural-scene Graz-02 object classes dataset.     

Shape-based indexing scheme for camera view invariant 3-D object retrieval

Camera view invariant 3-D object retrieval is an important issue in many traditional and emerging applications such as security, surveillance, computer-aided design (CAD), virtual reality, and place recognition. One straightforward method for camera view invariant 3-D object retrieval is to consider all the possible camera views of 3-D objects. However, capturing and maintaining such views require an enormous amount of time and labor. In addition, all camera views should be indexed for reasonable retrieval performance, which requires extra storage space and maintenance overhead. In the case of shape-based 3-D object retrieval, such overhead could be relieved by considering the symmetric shape feature of most objects. In this paper, we propose a new shape-based indexing and matching scheme of real or rendered 3-D objects for camera view invariant object retrieval. In particular, in order to remove redundant camera views to be indexed, we propose a camera view skimming scheme, which includes: i) mirror shape pairing and ii) camera view pruning according to the symmetrical patterns of object shapes. Since our camera view skimming scheme considerably reduces the number of camera views to be indexed, it could relieve the storage requirement and improve the matching speed without sacrificing retrieval accuracy. Through various experiments, we show that our proposed scheme can achieve excellent performance.     

Robust Object Detection with Interleaved Categorization and Segmentation

This paper presents a novel method for detecting and localizing objects of a visual category in cluttered real-world scenes. Our approach considers object categorization and figure-ground segmentation as two interleaved processes that closely collaborate towards a common goal. As shown in our work, the tight coupling between those two processes allows them to benefit from each other and improve the combined performance. The core part of our approach is a highly flexible learned representation for object shape that can combine the information observed on different training examples in a probabilistic extension of the Generalized Hough Transform. The resulting approach can detect categorical objects in novel images and automatically infer a probabilistic segmentation from the recognition result. This segmentation is then in turn used to again improve recognition by allowing the system to focus its efforts on object pixels and to discard misleading influences from the background. Moreover, the information from where in the image a hypothesis draws its support is employed in an MDL based hypothesis verification stage to resolve ambiguities between overlapping hypotheses and factor out the effects of partial occlusion. An extensive evaluation on several large data sets shows that the proposed system is applicable to a range of different object categories, including both rigid and articulated objects. In addition, its flexible representation allows it to achieve competitive object detection performance already from training sets that are between one and two orders of magnitude smaller than those used in comparable systems.     

Planar shape representation and matching under projective transformation

This paper introduces a new representation for planar objects which is invariant to projective transformation. Proposed representation relies on a new shape basis which we refer to as the conic basis. The conic basis takes conic-section coefficients as its dimensions and represents the object as a convex combination of conic-sections. Pairs of conic-sections in this new basis and their projective invariants provides the proposed view invariant representation. We hypothesize that two projectively transformed versions of an object result in the same representation. We show that our hypothesis provides promising recognition performance when we use the nearest neighbor rule to match projectively deformed objects.     

A CAD Model Based System for Object Recognition

3D object recognition is a difficult and yet an important problem in computer vision. A 3D object recognition system has two major components, namely: an object modeller and a system that performs the matching of stored representations to those derived from the sensed image. The performance of systems wherein the construction of object models is done by training from one or more images of the objects, has not been very satisfactory. Although objects used in a robotic workcell or in assembly processes have been designed using a CAD system, the vision systems used for recognition of these objects are independent of the CAD database. This paper proposes a scheme for interfacing the CAD database of objects and the computer vision processes used for recognising these objects. CAD models of objects are processed to generate vision oriented features that appear in the different views of the object and the same features are extracted from images of the object to identify the object and its pose.     

Visual object retrieval via block-based visual-pattern matching

This paper presents an object-based image retrieval using a method based on visual-pattern matching. A visual pattern is obtained by detecting the line edge from a square block using the moment-preserving edge detector. It is desirable and yet remains as a challenge for querying multimedia data by finding an object inside a target image. Given an object model, an added difficulty is that the object might be translated, rotated, and scaled inside a target image. Object segmentation and recognition is the primary step of computer vision for applying to image retrieval of higher-level image analysis. However, automatic segmentation and recognition of objects via object models is a difficult task without a priori knowledge about the shape of objects. Instead of segmentation and detailed object representation, the objective of this research is to develop and apply computer vision methods that explore the structure of an image object by visual-pattern detection to retrieve images from a database. A voting scheme based on generalized Hough transform is proposed to provide object search method, which is invariant to the translation, rotation, scaling of image data, and hence, invariant to orientation and position. Computer simulation results show that the proposed method gives good performance in terms of retrieval accuracy and robustness.