FORMS: A flexible object recognition and modelling system

We describe a flexible object recognition and modelling system (FORMS) which represents and recognizes animate objects from their silhouettes. This consists of a model for generating the shapes of animate objects which gives a formalism for solving the inverse problem of object recognition. We model all objects at three levels of complexity: (i) the primitives, (ii) the mid-grained shapes, which are deformations of the primitives, and (iii) objects constructed by using a grammar to join mid-grained shapes together. The deformations of the primitives can be characterized by principal component analysis or modal analysis. When doing recognition the representations of these objects are obtained in a bottom-up manner from their silhouettes by a novel method for skeleton extraction and part segmentation based on deformable circles. These representations are then matched to a database of prototypical objects to obtain a set of candidate interpretations. These interpretations are verified in a top-down process. The system is demonstrated to be stable in the presence of noise, the absence of parts, the presence of additional parts, and considerable variations in articulation and viewpoint. Finally, we describe how such a representation scheme can be automatically learnt from examples.     

Backpack: Detection of People Carrying Objects Using Silhouettes

We describe a video-rate surveillance algorithm for determining whether people are carrying objects or moving unencumbered from a stationary camera. The contribution of the paper is the shape analysis algorithm that both determines whether a person is carrying an object and segments the object from the person so that it can be tracked, e.g., during an exchange of objects between two people. As the object is segmented, an appearance model of the object is constructed. The method combines periodic motion estimation with static symmetry analysis of the silhouettes of a person in each frame of the sequence. Experimental results demonstrate robustness and real-time performance of the proposed algorithm.     

Path similarity skeleton graph matching

This paper presents a novel framework to for shape recognition based on object silhouettes. The main idea is to match skeleton graphs by comparing the shortest paths between skeleton endpoints. In contrast to typical tree or graph matching methods, we completely ignore the topological graph structure. Our approach is motivated by the fact that visually similar skeleton graphs may have completely different topological structures. The proposed comparison of shortest paths between endpoints of skeleton graphs yields correct matching results in such cases. The skeletons are pruned by contour partitioning with Discrete Curve Evolution, which implies that the endpoints of skeleton branches correspond to visual parts of the objects. The experimental results demonstrate that our method is able to produce correct results in the presence of articulations, stretching, and occlusion.     

The visual hull of smooth curved objects

The visual hull is a geometric entity that relates the shape of an object to its silhouettes or shadows. This paper develops the theory of the visual hull of generic smooth objects. We show that the visual hull can be constructed using surfaces which partition the viewpoint space of the aspect graph of the object. The surfaces are those generated by the visual events tangent crossing and triple point. An analysis based on the shape of the object at the tangency points of these surfaces allows pruning away many surfaces and patches not relevant to the construction. An algorithm for computing the visual hull is outlined.     

真实场景中基于体表示的目标外形和纹理获取

传统的基于图像的三维重建方法中,图像间的相关特征匹配是影响重建模型质量和精度的主要因素之一。提出一种基于体表示的真实目标的重建方法,回避了二维图像间的特征匹配问题,简化了建模的复杂度。该方法首先提取目标物的轮廓信息;其次单向单层遍历场景空间中目标物的最小三维包围盒,利用目标物固有的颜色信息不变性判断体元反向投影的一致性,从而获取目标的三维信息;最后利用获取的三维信息绘制得到真实目标的逼真模型。实验结果表明该方法建模精度较高,真实感强。     

Efficient Path-Space Differentiable Volume Rendering With Respect To Shapes

Differentiable rendering of translucent objects with respect to their shapes has been a long-standing problem. State-of-the-art methods require detecting object silhouettes or specifying change rates inside translucent objects—both of which can be expensive for translucent objects with complex shapes. In this paper, we address this problem for translucent objects with no refractive or reflective boundaries. By reparameterizing interior components of differential path integrals, our new formulation does not require change rates to be specified in the interior of objects. Further, we introduce new Monte Carlo estimators based on this formulation that do not require explicit detection of object silhouettes.     

Shape similarity measure based on correspondence of visual parts

A cognitively motivated similarity measure is presented and its properties are analyzed with respect to retrieval of similar objects in image databases of silhouettes of 2D objects. To reduce influence of digitization noise, as well as segmentation errors, the shapes are simplified by a novel process of digital curve evolution. To compute our similarity measure, we first establish the best possible correspondence of visual parts (without explicitly computing the visual parts). Then, the similarity between corresponding parts is computed and aggregated. We applied our similarity measure to shape matching of object contours in various image databases and compared it to well-known approaches in the literature. The experimental results justify that our shape matching procedure gives an intuitive shape correspondence and is stable with respect to noise distortions.     

基于Vega的舰船模型多尺度多视点提取技术

舰船三维目标识别是一个较为复杂的问题，需通过建立完整的、准确的三维舰船目标的二维视图库来描述一艘舰船目标，为此提出了一种基于Vega的舰船多尺度多视点提取技术。首先介绍了舰船目标的三维建模技术；然后研究了怎样由三维舰船模型经由Vega仿真软件自动批量的提取出任意尺度、任意视点的二维舰船视图，并实现自动将不同视点的显示转成二值图像存储。     

基于轮廓几何稀疏表示的刚性目标模型及其分级检测算法

刚性目标轮廓具有明显几何特性且不易受光照、纹理和颜色等因素影响.结合上述特性和图像稀疏表示原理,提出一种适用于刚性目标的分级检测算法.在基于部件模型(Part-based model, PBM)的框架下,采用匹配追踪算法将目标轮廓自适应地稀疏表示为几何部件的组合,根据部件与目标轮廓的匹配度,构建描述部件空间关系的有序链式结构.利用该链式结构的有序特性逐级缩小待检测范围,以匹配度为权值对各级部件显著图进行加权融合生成目标显著图. PASCAL图像库上的检测结果表明,该检测方法对具有显著轮廓特征的刚性目标有较好的检测结果,检测时耗较现有算法减少约60%~90%.     

Matching wire frame objects from their two dimensional perspective projections

A wire frame object consists of a set of three dimensional arcs, each arc being a sequence of conics and line segments lying in the same plane, with different arcs being allowed to lie on different planes. Given a picture taken by a camera focusing on one wire frame object, we show how to determine what the object is and where it is situated relative to the camera when the camera viewing parameters are unknown.

To accomplish the object identification, we begin with a segmented picture. Then we construct a ray from the lens to each point on the boundary of every region. For each region, the collection of its associated rays is a cone. We show that by constructing cones, the two-dimensional to three-dimensional matching problem is transformed into an equivalent three-dimensional to three-dimensional matching problem.

This matching problem is expressed as a nonlinear optimization search procedure on the 6 camera viewing parameters: the 3 translation parameters and the 3 rotation parameters. A solution is found when a viewing position and optical axis is determined which is consistent with the world knowledge we have of possible curves and the observed image data.     

Near-duplicate document image matching: A graphical perspective

A near-duplicate document image matching approach characterized by a graphical perspective is proposed in this paper. Document images are represented by graphs whose nodes correspond to the objects in the images. Consequently, the image matching problem is then converted to graph matching. To deal with the instability of object segmentation, a multi-granularity object tree is constructed for a document image. Each level in the tree corresponds to one possible object segmentation, while different levels are characterized by various object granularities. Some graphs can be generated from the tree and the objects associated with each graph may be of different granularities. Two graphs with the maximum similarity are found from the multi-granularity object trees of the two near-duplicate document images which are to be matched. The encouraging experimental results have demonstrated the effectiveness of the proposed approach.     

Curve and Surface Duals and the Recognition of Curved 3D Objects from their Silhouettes

This article addresses the problem of recognizing a solid bounded by a smooth surface in a single image. The proposed approach is based on a new representation for two- and three-dimensional shapes, called their signature, that exploits the close relationship between the dual of a surface and the dual of its silhouette in weak-perspective images. Objects are modeled by rotating them in front of a camera without any knowledge of or constraints on their motion. The signatures of their silhouettes are concatenated into a single object signature. To recognize an object from novel viewpoint other than those used during modeling, the signature of the contours extracted from a test photograph is matched to the signatures of all modeled objects signatures. This approach has been implemented, and recognition examples are presented.     

3D Free-Form Object Recognition Using Indexing by Contour Features

We address the problem of recognizing free-form 3D objects from a single 2D intensity image. A model-based solution within the alignment paradigm is presented which involves three major schemes—modeling, matching, and indexing. The modeling scheme constructs a set of model aspects which can predict the object contour as seen from any viewpoint. The matching scheme aligns the edgemap of a candidate model to the observed edgemap using an initial approximate pose. The major contribution of this paper involves the indexing scheme and its integration with modeling and matching to perform recognition. Indexing generates hypotheses specifying both candidate model aspects and approximate pose and scale. Hypotheses are ordered by likelihood based on prior knowledge of pre-stored models and the visual evidence from the observed objects. A prototype implementation has been tested in recognition and localization experiments with a database containing 658 model aspects from twenty 3D objects and eighty 2D objects. Bench tests and simulations show that many kinds of objects can be handled accurately and efficiently even in cluttered scenes. We conclude that the proposed recognition-by-alignment paradigm is a viable approach to many 3D object recognition problems.     

An iterative integrated framework for thermal–visible image registration,sensor fusion,and people tracking for video surveillance applications

In this work, we propose a new integrated framework that addresses the problems of thermal–visible video registration, sensor fusion, and people tracking for far-range videos. The video registration is based on a RANSAC trajectory-to-trajectory matching, which estimates an affine transformation matrix that maximizes the overlapping of thermal and visible foreground pixels. Sensor fusion uses the aligned images to compute sum-rule silhouettes, and then constructs thermal–visible object models. Finally, multiple object tracking uses blobs constructed in sensor fusion to output the trajectories. Results demonstrate the advantage of our proposed framework in obtaining better results for both image registration and tracking than separate image registration and tracking methods.     

Flexible syntactic matching of curves and its application toautomatic hierarchical classification of silhouettes

Curve matching is one instance of the fundamental correspondence problem. Our flexible algorithm is designed to match curves under substantial deformations and arbitrary large scaling and rigid transformations. A syntactic representation is constructed for both curves and an edit transformation which maps one curve to the other is found using dynamic programming. We present extensive experiments where we apply the algorithm to silhouette matching. In these experiments, we examine partial occlusion, viewpoint variation, articulation, and class matching (where silhouettes of similar objects are matched). Based on the qualitative syntactic matching, we define a dissimilarity measure and we compute it for every pair of images in a database of 121 images. We use this experiment to objectively evaluate our algorithm. First, we compare our results to those reported by others. Second, we use the dissimilarity values in order to organize the image database into shape categories. The veridical hierarchical organization stands as evidence to the quality of our matching and similarity estimation     

Distinctive Image Features from Scale-Invariant Keypoints

This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.     

Identification of space curves from two-dimensional perspective views

This paper describes a new method to be used for matching three-dimensional objects with curved surfaces to two-dimensional perspective views. The method requires for each three-dimensional object a stored model consisting of a closed space curve representing some characteristic connected curved edges of the object. The input is a two-dimensional perspective projection of one of the stored models represented by an ordered sequence of points. The input is converted to a spline representation which is sampled at equal intervals to derive a curvature function. The Fourier transform of the curvature function is used to represent the shape. The actual matching is reduced to a minimization problem which is handled by the Levenberg-Marquardt algorithm [3].     

Computational limitations of model-based recognition

Reliable object recognition is an essential part of most visual systems. Model-based approaches to object recognition use a database (a library) of modeled objects; for a given set of sensed data, the problem of model-based recognition is to identify and locate the objects from the library that are present in the data. We show that the complexity of model-based recognition depends very heavily on the number of object models in the library even if each object is modeled by a small number of discrete features. Specifically, deciding whether a discrete set of sensed data can be interpreted as transformed object models from a given library is NP-complete if the transformation is any combination of translation, rotation, scaling, and perspective projection. This suggests that efficient algorithms for model-based recognition must use additional structure to avoid the inherent computational difficulties. © 1998 John Wiley & Sons, Inc.