3D modeling of multiple-object scenes from sets of images

This paper proposes a new approach for multi-object 3D scene modeling. Scenes with multiple objects are characterized by object occlusions under several views, complex illumination conditions due to multiple reflections and shadows, as well as a variety of object shapes and surface properties. These factors raise huge challenges when attempting to model real 3D multi-object scene by using existing approaches which are designed mainly for single object modeling. The proposed method relies on the initialization provided by a rough 3D model of the scene estimated from the given set of multi-view images. The contributions described in this paper consists of two new methods for identifying and correcting errors in the reconstructed 3D scene. The first approach corrects the location of 3D patches from the scene after detecting the disparity between pairs of their projections into images. The second approach is called shape-from-contours and identifies discrepancies between projections of 3D objects and their corresponding contours, segmented from images. Both unsupervised and supervised segmentations are used to define the contours of objects.     

Distinctive Image Features from Scale-Invariant Keypoints

This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.     

On the Repeatability and Quality of Keypoints for Local Feature-based 3D Object Retrieval from Cluttered Scenes

3D object recognition from local features is robust to occlusions and clutter. However, local features must be extracted from a small set of feature rich keypoints to avoid computational complexity and ambiguous features. We present an algorithm for the detection of such keypoints on 3D models and partial views of objects. The keypoints are highly repeatable between partial views of an object and its complete 3D model. We also propose a quality measure to rank the keypoints and select the best ones for extracting local features. Keypoints are identified at locations where a unique local 3D coordinate basis can be derived from the underlying surface in order to extract invariant features. We also propose an automatic scale selection technique for extracting multi-scale and scale invariant features to match objects at different unknown scales. Features are projected to a PCA subspace and matched to find correspondences between a database and query object. Each pair of matching features gives a transformation that aligns the query and database object. These transformations are clustered and the biggest cluster is used to identify the query object. Experiments on a public database revealed that the proposed quality measure relates correctly to the repeatability of keypoints and the multi-scale features have a recognition rate of over 95% for up to 80% occluded objects.     

Representation and shape matching of 3-d objects

A three-dimensional scene analysis system for the shape matching of real world 3-D objects is presented. Various issues related to representation and modeling of 3-D objects are addressed. A new method for the approximation of 3-D objects by a set of planar faces is discussed. The major advantage of this method is that it is applicable to a complete object and not restricted to single range view which was the limitation of the previous work in 3-D scene analysis. The method is a sequential region growing algorithm. It is not applied to range images, but rather to a set of 3-D points. The 3-D model of an object is obtained by combining the object points from a sequence of range data images corresponding to various views of the object, applying the necessary transformations and then approximating the surface by polygons. A stochastic labeling technique is used to do the shape matching of 3-D objects. The technique matches the faces of an unknown view against the faces of the model. It explicitly maximizes a criterion function based on the ambiguity and inconsistency of classification. It is hierarchical and uses results obtained at low levels to speed up and improve the accuracy of results at higher levels. The objective here is to match the individual views of the object taken from any vantage point. Details of the algorithm are presented and the results are shown on several unknown views of a complicated automobile casting.     

The living in a lattice rule language

This paper presents a rule-based query language for an object-oriented database model. The database model supports complex objects, object identity, classes and types, and a class/type hierarchy. The instances are described by ‘object relations’ which are functions from a set of objects to value sets and other object sets. The rule language is based on object-terms which provide access to objects via the class hierarchy. Rules are divided into two classes: object-preserving rules manipulating existing objects (yielding a new ‘view’ on objects available in the object base) and object-generating rules creating new objects with properties derived from existing objects. The derived object sets are included in a class lattice. We give conditions for whether the instances of the ‘rules’ heads are ‘consistent’, i.e. represent object relations where the properties of the derived objects are functionally determined by the objects.     

Adaptive 3-D object recognition from multiple views

The authors address the problem of generating representations of 3-D objects automatically from exploratory view sequences of unoccluded objects. In building the models, processed frames of a video sequence are clustered into view categories called aspects, which represent characteristic views of an object invariant to its apparent position, size, 2-D orientation, and limited foreshortening deformation. The aspects as well as the aspect transitions of a view sequence are used to build (and refine) the 3-D object representations online in the form of aspect-transition matrices. Recognition emerges as the hypothesis that has accumulated the maximum evidence at each moment. The `winning' object continues to refine its representation until either the camera is redirected or another hypothesis accumulates greater evidence. This work concentrates on 3-D appearance modeling and succeeds under favorable viewing conditions by using simplified processes to segment objects from the scene and derive the spatial agreement of object features     

A tree-based context model for object recognition

There has been a growing interest in exploiting contextual information in addition to local features to detect and localize multiple object categories in an image. A context model can rule out some unlikely combinations or locations of objects and guide detectors to produce a semantically coherent interpretation of a scene. However, the performance benefit of context models has been limited because most of the previous methods were tested on data sets with only a few object categories, in which most images contain one or two object categories. In this paper, we introduce a new data set with images that contain many instances of different object categories, and propose an efficient model that captures the contextual information among more than a hundred object categories using a tree structure. Our model incorporates global image features, dependencies between object categories, and outputs of local detectors into one probabilistic framework. We demonstrate that our context model improves object recognition performance and provides a coherent interpretation of a scene, which enables a reliable image querying system by multiple object categories. In addition, our model can be applied to scene understanding tasks that local detectors alone cannot solve, such as detecting objects out of context or querying for the most typical and the least typical scenes in a data set.     

A CAD Model Based System for Object Recognition

3D object recognition is a difficult and yet an important problem in computer vision. A 3D object recognition system has two major components, namely: an object modeller and a system that performs the matching of stored representations to those derived from the sensed image. The performance of systems wherein the construction of object models is done by training from one or more images of the objects, has not been very satisfactory. Although objects used in a robotic workcell or in assembly processes have been designed using a CAD system, the vision systems used for recognition of these objects are independent of the CAD database. This paper proposes a scheme for interfacing the CAD database of objects and the computer vision processes used for recognising these objects. CAD models of objects are processed to generate vision oriented features that appear in the different views of the object and the same features are extracted from images of the object to identify the object and its pose.     

Collision-free object movement using vector fields

Presents a technique for automatically providing animation and collision avoidance in a general-purpose computer graphics system. The technique, which relies on an expanded notion of vector fields, allows users to set up and animate objects easily, then prevents objects from colliding as the animation proceeds. This technique automatically generates volume octree vector fields around objects in a scene. These vector fields affect object motion and animation, and also provide for automatic collision avoidance for arbitrary objects. Applications of collision avoidance in an animation system encompass any scene containing object movement above or around other objects     

Visual learning and recognition of 3-d objects from appearance

The problem of automatically learning object models for recognition and pose estimation is addressed. In contrast to the traditional approach, the recognition problem is formulated as one of matching appearance rather than shape. The appearance of an object in a two-dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties and constant for a rigid object, pose and illumination vary from scene to scene. A compact representation of object appearance is proposed that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a manifold. Given an unknown input image, the recognition system projects the image to eigenspace. The object is recognized based on the manifold it lies on. The exact position of the projection on the manifold determines the object's pose in the image.A variety of experiments are conducted using objects with complex appearance characteristics. The performance of the recognition and pose estimation algorithms is studied using over a thousand input images of sample objects. Sensitivity of recognition to the number of eigenspace dimensions and the number of learning samples is analyzed. For the objects used, appearance representation in eigenspaces with less than 20 dimensions produces accurate recognition results with an average pose estimation error of about 1.0 degree. A near real-time recognition system with 20 complex objects in the database has been developed. The paper is concluded with a discussion on various issues related to the proposed learning and recognition methodology.     

Detecting moving objects using the rigidity constraint

A method for visually detecting moving objects from a moving camera using point correspondences in two orthographic views is described. The method applies a simple structure-from-motion analysis and then identifies those points inconsistent with the interpretation of the scene as a single rigid object. It is effective even when the actual motion parameters cannot be recovered. Demonstrations are presented using point correspondences automatically determined from real image sequences     

Multilayer background modeling under occlusions

A multilayer background modeling technique is presented for video surveillance. Rather than simply classifying all features in a scene as either dynamically moving foreground or long-lasting, stationary background, a temporal model is used to place each scene object in time relative to each other. Foreground objects that become stationary are registered as layers on top of the background layer. In this process of layer formation, the algorithm deals with ”fake objects” created by moved background, and noise created by dynamic background and moving foreground objects. Objects that leave the scene are removed based on the occlusion reasoning among layers. The technique allows us to understand and visualize a scene with multiple objects entering, leaving, and occluding each other at different points in time. This scene understanding leads to a richer representation of temporal scene events than traditional foreground/background segmentation. The technique builds on a low-cost background modeling technique that makes it suitable for embedded, real-time platforms.     

Genetic object recognition using combinations of views

Investigates the application of genetic algorithms (GAs) for recognizing real 2D or 3D objects from 2D intensity images, assuming that the viewpoint is arbitrary. Our approach is model-based (i.e. we assume a pre-defined set of models), while our recognition strategy relies on the theory of algebraic functions of views. According to this theory, the variety of 2D views depicting an object can be expressed as a combination of a small number of 2D views of the object. This implies a simple and powerful strategy for object recognition: novel 2D views of an object (2D or 3D) can be recognized by simply matching them to combinations of known 2D views of the object. In other words, objects in a scene are recognized by "predicting" their appearance through the combination of known views of the objects. This is an important idea, which is also supported by psychophysical findings indicating that the human visual system works in a similar way. The main difficulty in implementing this idea is determining the parameters of the combination of views. This problem can be solved either in the space of feature matches among the views ("image space") or the space of parameters ("transformation space"). In general, both of these spaces are very large, making the search very time-consuming. In this paper, we propose using GAs to search these spaces efficiently. To improve the efficiency of genetic searching in the transformation space, we use singular value decomposition and interval arithmetic to restrict the genetic search to the most feasible regions of the transformation space. The effectiveness of the GA approaches is shown on a set of increasingly complex real scenes where exact and near-exact matches are found reliably and quickly     

Linear object classes and image synthesis from a single exampleimage

The need to generate new views of a 3D object from a single real image arises in several fields, including graphics and object recognition. While the traditional approach relies on the use of 3D models, simpler techniques are applicable under restricted conditions. The approach exploits image transformations that are specific to the relevant object class, and learnable from example views of other “prototypical” objects of the same class. In this paper, we introduce such a technique by extending the notion of linear class proposed by the authors (1992). For linear object classes, it is shown that linear transformations can be learned exactly from a basis set of 2D prototypical views. We demonstrate the approach on artificial objects and then show preliminary evidence that the technique can effectively “rotate” high-resolution face images from a single 2D view     

Delta abstractions: A technique for managing database states in runtime debugging of active database rules

Delta abstractions are introduced as a mechanism for managing database states during the execution of active database rules. Delta abstractions build upon the use of object deltas, capturing changes to individual objects through a system-supported, collapsible type structure. The object delta structure is implemented using object-oriented concepts such as encapsulation and inheritance so that all database objects inherit the ability to transparently create and manage delta values. Delta abstractions provide an additional layer to the database programmer for organizing object deltas according to different language components that induce database changes, such as methods and active rules. As with object deltas, delta abstractions are transparently created and maintained by the active database system. We define different types of delta abstractions as views of object deltas and illustrate how the services of delta abstractions can be used to inspect the state of active rule execution. An active rule analysis and debugging tool has been implemented to demonstrate the use of object deltas and delta abstractions for dynamic analysis of active rules at runtime.     

Imitating Popular Photos to Select Views for an Indoor Scene

Selecting informative and visually appealing views for 3D indoor scenes is beneficial for the housing, decoration, and entertainment industries. A set of views that exhibit comfort, aesthetics, and functionality of a particular scene can attract customers and facilitate business transactions. However, selecting views for an indoor scene is challenging because the system has to consider not only the need to reveal as much information as possible, but also object arrangements, occlusions, and characteristics. Since there can be many principles utilized to guide the view selection, and various principles to follow under different circumstances, we achieve the goal by imitating popular photos on the Internet. Specifically, we select the view that can optimize the contour similarity of corresponding objects to the photo. Because the selected view can be inadequate if object arrangements in the 3D scene and the photo are different, our system imitates many popular photos and selects a certain number of views. After that, it clusters the selected views and determines the view/cluster centers by the weighted average to finally exhibit the scene. Experimental results demonstrate that the views selected by our method are visually appealing.     

Application of differential geometry to recognize and locate partially occluded objects

A new method has been proposed to recognize and locate partially occluded two-dimensional rigid objects of a given scene. For this purpose we initially generate a set of local features of the shapes using the concept of differential geometry. Finally a computer vision scheme, based upon matching local features of the objects in a scene and the models which are considered as cognitive database, is described using hypothesis generation and verification of features for the best possible recognition.     

Extraction of rules from natural objects for automated mechanicalprocessing

In process applications, fast and accurate extraction of complex information from an object for the purpose of mechanical processing of that object, is often required. In this paper, a general rule-based approach is developed using a database of measurable geometric “features” and associated complex information. The rules relate the features to the complex processing information. During the on-line processing, the object features are measured and passed into the rule base. The output from the rule base is the complex information that is needed to process the object. A methodology is developed to generate probabilistic rules for the rule base using multivariate probability densities. A knowledge integration scheme is also developed which combines statistical knowledge with expert knowledge in order to improve the reliability and efficiency of information extraction. The rule generation methodology is implemented in a knowledge-based vision system for process information recognition. As an illustrative example, the problem of efficient head removal in an automated salmon processing plant is considered     

基于SIFT算法的目标特征检测与提取技术研究

验证了一种能够在不同图像之间进行同一个物体相匹配的方法,具有很强的可靠性,称之为SIFT算法(尺度不变特征变换).SIFT算法能够处理图像间发生的尺度变换、旋转、很大范围内的仿射形变、视角变换、噪声以及光照变换.它的功能十分强大,甚至可以仅仅根据一个简单的物体特征,在一个大型数据库中的许多高品质图像中进行相应目标的寻找...