3D scene retrieval and recognition with Depth Gradient Images

The intention of the strategy proposed in this paper is to solve the object retrieval problem in highly complex scenes using 3D information. In the worst case scenario the complexity of the scene includes several objects with irregular or free-form shapes, viewed from any direction, which are self-occluded or partially occluded by other objects with which they are in contact and whose appearance is uniform in intensity/color. This paper introduces and analyzes a new 3D recognition/pose strategy based on DGI (Depth Gradient Images) models. After comparing it with current representative techniques, we can affirm that DGI has very interesting prospects.The DGI representation synthesizes both surface and contour information, thus avoiding restrictions concerning the layout and visibility of the objects in the scene. This paper first explains the key concepts of the DGI representation and shows the main properties of this method in comparison to a set of known techniques. The performance of this strategy in real scenes is then reported. Details are also presented of a wide set of experimental tests, including results under occlusion, performance with injected noise and experiments with cluttered scenes of a high level of complexity.     

Point Signatures: A New Representation for 3D Object Recognition

Few systems capable of recognizing complex objects with free-form (sculptured) surfaces have been developed. The apparent lack of success is mainly due to the lack of a competent modelling scheme for representing such complex objects. In this paper, a new form of point representation for describing 3D free-form surfaces is proposed. This representation, which we call the point signature, serves to describe the structural neighbourhood of a point in a more complete manner than just using the 3D coordinates of the point. Being invariant to rotation and translation, the point signature can be used directly to hypothesize the correspondence to model points with similar signatures. Recognition is achieved by matching the signatures of data points representing the sensed surface to the signatures of data points representing the model surface.The use of point signatures is not restricted to the recognition of a single-object scene to a small library of models. Instead, it can be extended naturally to the recognition of scenes containing multiple partially-overlapping objects (which may also be juxtaposed with each other) against a large model library. No preliminary phase of segmenting the scene into the component objects is required. In searching for the appropriate candidate model, recognition need not proceed in a linear order which can become prohibitive for a large model library. For a given scene, signatures are extracted at arbitrarily spaced seed points. Each of these signatures is used to vote for models that contain points having similar signatures. Inappropriate models with low votes can be rejected while the remaining candidate models are ordered according to the votes they received. In this way, efficient verification of the hypothesized candidates can proceed by testing the most likely model first. Experiments using real data obtained from a range finder have shown fast recognition from a library of fifteen models whose complexities vary from that of simple piecewise quadric shapes to complicated face masks. Results from the recognition of both single-object and multiple-object scenes are presented.     

三维物体识别研究进展

出于工业和医疗等领域大量现实应用的需要,如今三维物体识别已成为一个很活跃的研究领域。一般来说,三维物体识别系统可以通过两个阶段的处理来完成三维物体的识别和定位,首先用传感器获取的场景输入数据来得到场景的表达;然后将它与数据库中存储的物体表达相匹配。为了推动该领域研究进一步发展,因而对近１０ａ年中该识别过程中必须解决的感传器类型、三维物体表达方法和匹配策略等３个方面问题的研究成果进行了综述,对主要方法进行分类和总结;并提出了一些三维视觉系统中还需要深入研究的问题,包括对所研究物体形状的限制、复杂背景的影响和表达以及识别中的“整体和局部”的矛盾等。     

3D Free-Form Object Recognition Using Indexing by Contour Features

We address the problem of recognizing free-form 3D objects from a single 2D intensity image. A model-based solution within the alignment paradigm is presented which involves three major schemes—modeling, matching, and indexing. The modeling scheme constructs a set of model aspects which can predict the object contour as seen from any viewpoint. The matching scheme aligns the edgemap of a candidate model to the observed edgemap using an initial approximate pose. The major contribution of this paper involves the indexing scheme and its integration with modeling and matching to perform recognition. Indexing generates hypotheses specifying both candidate model aspects and approximate pose and scale. Hypotheses are ordered by likelihood based on prior knowledge of pre-stored models and the visual evidence from the observed objects. A prototype implementation has been tested in recognition and localization experiments with a database containing 658 model aspects from twenty 3D objects and eighty 2D objects. Bench tests and simulations show that many kinds of objects can be handled accurately and efficiently even in cluttered scenes. We conclude that the proposed recognition-by-alignment paradigm is a viable approach to many 3D object recognition problems.     

Shape spectrum based view grouping and matching of 3D free-formobjects

We address the problem of constructing view aspects of 3D free-form objects for efficient matching during recognition. We introduce a novel view representation based on “shape spectrum” features, and propose a general and powerful technique for organizing multiple views of objects of complex shape and geometry into compact and homogeneous clusters. Our view grouping technique obviates the need for surface segmentation and edge detection. Experiments on 6,400 synthetically generated views of 20 free-form objects and 100 real range images of 10 sculpted objects demonstrate the good performance of our shape spectrum based model view selection technique     

Photogrammetric reconstruction of free-form objects with curvilinear structures

The shapes of many natural or man-made objects have curve features. The images of such curves usually do not have sufficient distinctive features to apply conventional feature-based reconstruction algorithms. In this paper, we introduce a photogrammetric method for recovering free-form objects with curvilinear structures. Our method chooses to obtain the topology and geometry of a sparse 3D wireframe of the object first instead of directly recovering a surface or volume model. Surface patches covering the object are then constructed to interpolate the curves in this wireframe while satisfying certain heuristics such as minimal bending energy. The result is an object surface model with curvilinear structures from a sparse set of images. We can produce realistic texture-mapped renderings of the object model from arbitrary viewpoints. Reconstruction results on multiple real objects are presented to demonstrate the effectiveness of our approach.     

Surface signatures: an orientation independent free-form surface representation scheme for the purpose of objects registration and matching

This paper introduces anew free-form surface representation scheme for the purpose of fast and accurate registration and matching. Accurate registration of surfaces is a common task in computer vision. The proposed representation scheme captures the surface curvature information (seen from certain points) and produces images, called "surface signatures," at these points. Matching signatures of different surfaces enables the recovery of the transformation parameters between these surfaces. We propose using template matching to compare the signature images. To enable partial matching, another criterion, the overlap ratio is used. This representation scheme can be used as a global representation of the surface as well as a local one and performs near real-time registration. We show that the signature representation can be used to recover scaling transformation as well as matching objects in 3D scenes in the presence of clutter and occlusion. Applications presented include: free-form object matching, multimodal medical volumes registration, and dental teeth reconstruction from intraoral images.     

Free-form geometric modeling by integrating parametric and implicit PDEs

Parametric PDE techniques, which use partial differential equations (PDEs) defined over a 2D or 3D parametric domain to model graphical objects and processes, can unify geometric attributes and functional constraints of the models. PDEs can also model implicit shapes defined by level sets of scalar intensity fields. In this paper, we present an approach that integrates parametric and implicit trivariate PDEs to define geometric solid models containing both geometric information and intensity distribution subject to flexible boundary conditions. The integrated formulation of second-order or fourth-order elliptic PDEs permits designers to manipulate PDE objects of complex geometry and/or arbitrary topology through direct sculpting and free-form modeling. We developed a PDE-based geometric modeling system for shape design and manipulation of PDE objects. The integration of implicit PDEs with parametric geometry offers more general and arbitrary shape blending and free-form modeling for objects with intensity attributes than pure geometric models     

深度图像中基于轮廓曲线和局部区域特征的3维物体识别

为了提高3维物体目标识别系统的性能及降低计算复杂度,提出一种由粗到细的识别方法。该方法利用深度图像所提供的信息,分两步完成识别过程。首先基于轮廓曲线计算其特征点,并映射到原有轮廓空间,以标志点序列表征原由轮廓进行匹配,在识别初期迅速排除模型库中不相似目标和差异较大的姿态,生成目标候选列表用于精确匹配,以提高识别效率。精确匹配采用一种基于局部区域特征的识别方法,以投票的策略获取最佳结果。局部区域由SIFT算子确定位置和数量,区域特征主要由表面指数和法向量夹角组成,具有平移和旋转不变性。为了更进一步提高效率和降低存储空间,模型库的数据分为轮廓和表面信息两部分,分别以标志位序列和哈希表的形式存储。实验结果表明,该方法具有良好的实时性和识别率,对遮挡和干扰有一定的适应性。     

3D sketching for aesthetic design using fully free-form deformation features

This paper addresses the designers’ activity and in particular the way designers express an object shape in 2D sketches through character lines and how these lines form a basis for sketching shapes in 3D. The tools currently available in commercial CAS/CAD systems to manipulate the digital models are still not sufficiently suited to support design. In this paper, the so-called fully free-form deformation features (δ-F⁴) are introduced as a modelling method to take into account the curve-oriented stylists’ way of working. Both the advantages of a free-form surface deformation method and a feature-based approach are merged to define these high-level modelling entities allowing for a direct manipulation of surfaces through a limited number of intuitive parameters. Such features incorporate several characteristics designed to handle the uncertainties and/or inconsistencies of the designer's input during a sketching activity. In addition, a δ-F⁴ classification is proposed to enable a fast access to the desired shape according to its semantics and characteristics.     

Non-rigid 3D shape tracking from multiview video

We present a fast and efficient non-rigid shape tracking method for modeling dynamic 3D objects from multiview video. Starting from an initial mesh representation, the shape of a dynamic object is tracked over time, both in geometry and topology, based on multiview silhouette and 3D scene flow information. The mesh representation of each frame is obtained by deforming the mesh representation of the previous frame towards the optimal surface defined by the time-varying multiview silhouette information with the aid of 3D scene flow vectors. The whole time-varying shape is then represented as a mesh sequence which can efficiently be encoded in terms of restructuring and topological operations, and small-scale vertex displacements along with the initial model. The proposed method has the ability to deal with dynamic objects that may undergo non-rigid transformations and topological changes. The time-varying mesh representations of such non-rigid shapes, which are not necessarily of fixed connectivity, can successfully be tracked thanks to restructuring and topological operations employed in our deformation scheme. We demonstrate the performance of the proposed method both on real and synthetic sequences.     

Vision-based global localization for mobile robots with hybrid maps of objects and spatial layouts

This paper presents a novel vision-based global localization that uses hybrid maps of objects and spatial layouts. We model indoor environments with a stereo camera using the following visual cues: local invariant features for object recognition and their 3D positions for object pose estimation. We also use the depth information at the horizontal centerline of image where the optical axis passes through, which is similar to the data from a 2D laser range finder. This allows us to build our topological node that is composed of a horizontal depth map and an object location map. The horizontal depth map describes the explicit spatial layout of each local space and provides metric information to compute the spatial relationships between adjacent spaces, while the object location map contains the pose information of objects found in each local space and the visual features for object recognition. Based on this map representation, we suggest a coarse-to-fine strategy for global localization. The coarse pose is estimated by means of object recognition and SVD-based point cloud fitting, and then is refined by stochastic scan matching. Experimental results show that our approaches can be used for an effective vision-based map representation as well as for global localization methods.     

Curve and Surface Duals and the Recognition of Curved 3D Objects from their Silhouettes

This article addresses the problem of recognizing a solid bounded by a smooth surface in a single image. The proposed approach is based on a new representation for two- and three-dimensional shapes, called their signature, that exploits the close relationship between the dual of a surface and the dual of its silhouette in weak-perspective images. Objects are modeled by rotating them in front of a camera without any knowledge of or constraints on their motion. The signatures of their silhouettes are concatenated into a single object signature. To recognize an object from novel viewpoint other than those used during modeling, the signature of the contours extracted from a test photograph is matched to the signatures of all modeled objects signatures. This approach has been implemented, and recognition examples are presented.     

Symbolic signatures for deformable shapes

Recognizing classes of objects from their shape is an unsolved problem in machine vision that entails the ability of a computer system to represent and generalize complex geometrical information on the basis of a finite amount of prior data. A practical approach to this problem is particularly difficult to implement, not only because the shape variability of relevant object classes is generally large, but also because standard sensing devices used to capture the real world only provide a partial view of a scene, so there is partial information pertaining to the objects of interest. In this work, we develop an algorithmic framework for recognizing classes of deformable shapes from range data. The basic idea of our component-based approach is to generalize existing surface representations that have proven effective in recognizing specific 3D objects to the problem of object classes using our newly introduced symbolic-signature representation that is robust to deformations, as opposed to a numeric representation that is often tied to a specific shape. Based on this approach, we present a system that is capable of recognizing and classifying a variety of object shape classes from range data. We demonstrate our system in a series of large-scale experiments that were motivated by specific applications in scene analysis and medical diagnosis.     

Free-Form Simulation of Sequential Etching and Surface Characterization for 3-D MEMS Fabrication

A new diffusion-based simulation model of isotropic wet etching and free-form surface characterization method for 3-D free-form microelectromechanical systems (MEMS) fabrication is presented in this paper. To simulate the etching process, a diffusion-based model solved by the finite-element method (FEM) has been developed, allowing extraction of more accurate etch-front data at discrete time steps. In the developed method, free-form MEMS objects are modeled as B-spline functions with material concentration. Finite elements are generated by discretization in the parametric domain of the free-form object and mapping back to the Euclidean space. Points on the etch front are extracted using a Z-map method. The extracted point data are characterized to obtain a B-spline representation of the etch-front surface. Examples from the isotropic etching simulation of 2-D and 3-D objects with both regular and free-form geometry are presented. The developed method allows the simulation of 3-D objects with free-form input and free-form mask opening and facilitates the simulation of sequential etching of free-form objects with irregular mask openings. This paper also discusses applications of the developed method in MEMS process planning that can be realized by taking advantage of the better control of geometry that it provides in MEMS fabrication.     

A CAD Model Based System for Object Recognition

3D object recognition is a difficult and yet an important problem in computer vision. A 3D object recognition system has two major components, namely: an object modeller and a system that performs the matching of stored representations to those derived from the sensed image. The performance of systems wherein the construction of object models is done by training from one or more images of the objects, has not been very satisfactory. Although objects used in a robotic workcell or in assembly processes have been designed using a CAD system, the vision systems used for recognition of these objects are independent of the CAD database. This paper proposes a scheme for interfacing the CAD database of objects and the computer vision processes used for recognising these objects. CAD models of objects are processed to generate vision oriented features that appear in the different views of the object and the same features are extracted from images of the object to identify the object and its pose.