Model-based recognition of 3D objects from single images

In this work, we treat major problems of object recognition which have received relatively little attention lately. Among them are the loss of depth information in the projection from a 3D object to a single 2D image, and the complexity of finding feature correspondences between images. We use geometric invariants to reduce the complexity of these problems. There are no geometric invariants of a projection from 3D to 2D. However, given certain modeling assumptions about the 3D object, such invariants can be found. The modeling assumptions can be either a particular model or a generic assumption about a class of models. Here, we use such assumptions for single-view recognition. We find algebraic relations between the invariants of a 3D model and those of its 2D image under general projective projection. These relations can be described geometrically as invariant models in a 3D invariant space, illuminated by invariant “light rays,” and projected onto an invariant version of the given image. We apply the method to real images     

3D Motion Estimation and Motion Fusion by Affine Region Matching

In this paper,a new method is presented for 3D motion estimation by image region correspondences using stereo cameras.Under the weak perspectivity assumption.we first employ the moment tensor theory (Cyganski and Orr^[11]) to compute the monocular affine transformations relating images taken by the same camera at different time instants and the binocular affine transformations relating images taken by different cameras at the same time instant.We then show that 3D motion can be recovered from these 2D transformations.A space-time fusion strategy is proposed to aim at robust results.No knowledge of point correspondences if requred in the above processes and the computations involved are linear.To find corresponding image regions,new affine invariants,which show stronger invariance,are derived in term of tensor contraction theory.Experiments on real motion images are conducted to verify the proposed method.     

3D object recognition using invariants of 2D projection curves

This paper presents a new method for recognizing 3D objects based on the comparison of invariants of their 2D projection curves. We show that Euclidean equivalent 3D surfaces imply affine equivalent 2D projection curves that are obtained from the projection of cross-section curves of the surfaces onto the coordinate planes. Planes used to extract cross-section curves are chosen to be orthogonal to the principal axes of the defining surfaces. Projection curves are represented using implicit polynomial equations. Affine algebraic and geometric invariants of projection curves are constructed and compared under a variety of distance measures. Results are verified by several experiments with objects from different classes and within the same class.     

Foundations of semi-differential invariants

This paper elaborates the theoretical foundations of a semi-differential framework for invariance. Semi-differential invariants combine coordinates and their derivatives with respect to some contour parameter at several points of the image contour, thus allowing for an optimal trade-off between identification of points and the calculation of derivatives. A systematic way of generating complete and independent sets of such invariants is presented. It is also shown that invariance under reparametrisation can be cast in the same framework. The theory is illustrated by a complete analysis of 2D affine transformations. In a companion paper (Pauwels et al. 1995) these affine semi-differential invariants are implemented in the computer program FORM (Flat Object Recognition Method) for the recognition of planar contours under pseudo-perspective projection.Post-doctoral Research Fellow of the Belgian National Fund for Scientific Research (N.F.W.O.).     

A curve fitting problem and its application in modeling objects inmonocular image sequences

Presents a solution to a particular curve (surface) fitting problem and demonstrate its application in modeling objects from monocular image sequences. The curve-fitting algorithm is based on a modified nonparametric regression method, which forms the core contribution of this work. This method is far more effective compared to standard estimation techniques, such as the maximum likelihood estimation method, and can take into account the discontinuities present in the curve. Next, the theoretical results of this 1D curve estimation technique ate extended significantly for an object modeling problem. The input to the algorithm is a monocular image sequence of an object undergoing rigid motion. By using the affine camera projection geometry and a given choice of an image frame pair in the sequence, we adopt the KvD (Koenderink and van Doorn, 1991) model to express the depth at each point on the object as a function of the unknown out-of-plane rotation, and some measurable quantities computed directly from the optical flow. This is repeated for multiple image pairs (keeping one fixed image frame which we formally call the base image and choosing another frame from the sequence). The depth map is next estimated from these equations using the modified nonparametric regression analysis. We conducted experiments on various image sequences to verify the effectiveness of the technique. The results obtained using our curve-fitting technique can be refined further by hierarchical techniques, as well as by nonlinear optimization techniques in structure from motion     

Solving the recognition problem for six lines using the Dixon resultant

The “Six-line Problem” arises in computer vision and in the automated analysis of images. Given a three-dimensional (3D) object, one extracts geometric features (for example six lines) and then, via techniques from algebraic geometry and geometric invariant theory, produces a set of 3D invariants that represents that feature set. Suppose that later an object is encountered in an image (for example, a photograph taken by a camera modeled by standard perspective projection, i.e. a “pinhole” camera), and suppose further that six lines are extracted from the object appearing in the image. The problem is to decide if the object in the image is the original 3D object. To answer this question two-dimensional (2D) invariants are computed from the lines in the image. One can show that conditions for geometric consistency between the 3D object features and the 2D image features can be expressed as a set of polynomial equations in the combined set of two- and three-dimensional invariants. The object in the image is geometrically consistent with the original object if the set of equations has a solution. One well known method to attack such sets of equations is with resultants. Unfortunately, the size and complexity of this problem made it appear overwhelming until recently. This paper will describe a solution obtained using our own variant of the Cayley–Dixon–Kapur–Saxena–Yang resultant. There is reason to believe that the resultant technique we employ here may solve other complex polynomial systems.     

结合轮廓与形状特征的仿射形状匹配

目的 针对仿射变换下形状匹配中存在的描述子对形状的描述能力不足,以及描述子计算耗时大的问题,改进基于所有图像点投影的方法,提出一种利用轮廓计算投影面积的仿射形状匹配算法。方法 该算法分为粗匹配和精匹配两个阶段。粗匹配阶段以CSS角点作为备选特征点,首先统计轮廓投影面积分布作为特征点描述子;然后利用动态规划蚁群算法匹配两幅图片公共特征点序列,并将匹配好的特征点序列记为对应的新特征点;最后采用该新特征点划分目标曲线,得到对应的轮廓曲线;这一阶段的目的是对形状的筛选以及寻找一致的轮廓特征点,同时完成轮廓曲线的划分。精匹配阶段,采用小波仿射不变描述子,对粗匹配阶段匹配代价最小的5%的目标进行对应曲线匹配,得到精匹配阶段的匹配代价,从而实现对仿射目标的识别;精匹配弥补了描述子对轮廓细节描述不足的问题。结果 算法的平均检索速度比传统基于形状投影分布描述子提高44.3%,在MPEG-7图像库上的检索效果为98.65%,在MPEG-7仿射图像库上的查准率与查全率综合评价指标比传统的基于形状投影分布描述子高3.1%,比形状上下文高25%。结论 本文算法匹配效果好,效率高,抗噪性强,解决了仿射描述子计算速度慢、描述能力不足的问题,能有效地应用于仿射形状匹配与检索领域。     

Geometric and illumination invariants for object recognition

We propose invariant formulations that can potentially be combined into a single system. In particular, we describe a framework for computing invariant features which are insensitive to rigid motion, affine transform, changes of parameterization and scene illumination, perspective transform, and view point change. This is unlike most current research on image invariants which concentrates on either geometric or illumination invariants exclusively. The formulations are widely applicable to many popular basis representations, such as wavelets, short-time Fourier analysis, and splines. Exploiting formulations that examine information about shape and color at different resolution levels, the new approach is neither strictly global nor local. It enables a quasi-localized, hierarchical shape analysis which is rarely found in other known invariant techniques, such as global invariants. Furthermore, it does not require estimating high-order derivatives in computing invariants (unlike local invariants), whence is more robust. We provide results of numerous experiments on both synthetic and real data to demonstrate the validity and flexibility of the proposed framework     

Matching 3-D Models to 2-D Images

We consider the problem of analytically characterizing the set of all 2-D images that a group of 3-D features may produce, and demonstrate that this is a useful thing to do. Our results apply for simple point features and point features with associated orientation vectors when we model projection as a 3-D to 2-D affine transformation. We show how to represent the set of images that a group of 3-D points produces with two lines (1-D subspaces), one in each of two orthogonal, high-dimensional spaces, where a single image group corresponds to one point in each space. The images of groups of oriented point features can be represented by a 2-D hyperbolic surface in a single high-dimensional space. The problem of matching an image to models is essentially reduced to the problem of matching a point to simple geometric structures. Moreover, we show that these are the simplest and lowest dimensional representations possible for these cases.We demonstrate the value of this way of approaching matching by applying our results to a variety of vision problems. In particular, we use this result to build a space-efficient indexing system that performs 3-D to 2-D matching by table lookup. This system is analytically built and accessed, accounts for the effects of sensing error, and is tested on real images. We also derive new results concerning the existence of invariants and non-accidental properties in this domain. Finally, we show that oriented points present unexpected difficulties: indexing requires fundamentally more space with oriented than with simple points, we must use more images in a motion sequence to determine the affine structure of oriented points, and the linear combinations result does not hold for oriented points.     

基于组合不变矩和神经网络的三维物体识别

在三维物体识别系统中,提出将三维物体的Hu不变矩和仿射不变矩两者的低阶矩组合作为三维物体的特征,结合改进的BP神经网络应用于三维物体的分类识别。理论分析和仿真实验表明组合这两种矩特征进行物体识别,性能优于单独使用Hu不变矩,如果进一步对这两种组合的矩特征进行主成分分析处理,可显著提高系统识别性能,并减少网络的训练时间。     

Statistical cue integration in DAG deformable models

Deformable models are a useful modeling paradigm in computer vision. A deformable model is a curve, a surface, or a volume, whose shape, position, and orientation are controlled through a set of parameters. They can represent manufactured objects, human faces and skeletons, and even bodies of fluid. With low-level computer vision and image processing techniques, such as optical flow, we extract relevant information from images. Then, we use this information to change the parameters of the model iteratively until we find a good approximation of the object in the images. When we have multiple computer vision algorithms providing distinct sources of information (cues), we have to deal with the difficult problem of combining these, sometimes conflicting contributions in a sensible way. In this paper, we introduce the use of a directed acyclic graph (DAG) to describe the position and Jacobian of each point of deformable models. This representation is dynamic, flexible, and allows computational optimizations that would be difficult to do otherwise. We then describe a new method for statistical cue integration method for tracking deformable models that scales well with the dimension of the parameter space. We use affine forms and affine arithmetic to represent and propagate the cues and their regions of confidence. We show that we can apply the Lindeberg theorem to approximate each cue with a Gaussian distribution, and can use a maximum-likelihood estimator to integrate them. Finally, we demonstrate the technique at work in a 3D deformable face tracking system on monocular image sequences with thousands of frames.     

Affine Curvature Scale Space with Affine Length Parametrisation

The maxima of Curvature Scale Space (CSS) image have been used to represent 2D shapes under affine transforms. The CSS image is expected to be in the MPEG-7 package of standards. Since the CSS image employs the arc length parametrisation which is not affine invariant, we expect some deviations in the maxima of the CSS image under general affine transforms. Affine length and affine curvature have already been introduced and used as alternatives to arc length and conventional curvature in affine transformed environments. The utility of using these parameters to enrich the CSS representation is addressed in this paper. We use arc length to parametrise the curve prior to computing its CSS image. The parametrisation has been proven to be invariant under affine transformation and has been used in many affine invariant shape recognition methods. Since the organisation of the CSS image is based on curvature zero crossings of the curve, in this paper, we also investigate the advantages and shortcomings of using affine curvature in computation of the CSS image. The enriched CSS representations are then used to find similar shapes from a very large prototype database, and also a small classified database, both consisting of original as well as affine transformed shapes. An improvement is observed over the conventional CSS image.     

An automatic method for generating affine moment invariants

Affine moment invariants are important if one wants to recognize the surface of a plane in three dimensions when the orientation of the plane is not known beforehand and only two-dimensional information is available. The notion of generating function is introduced as a simple and straightforward way to derive various affine invariants. By this notion, we can get the explicit construction of much more affine moment invariants. Based on this conclusion, a large set of invariant polynomials can be generated automatically and immediately by the algorithm we have designed. These new affine moment invariants can be applied to recognize the image. Approaches in this paper will improve the practicability of affine invariants in object recognition applications.     

An Approach to the Vanishing Line Identification Based on Normalized Barycentric Coordinates

The vanishing line is useful information for recovering affine properties of the plane in computer vision. This paper describes how to determine analytically the vanishing line from a single perspective view of a plane containing the four points of known normalized barycentric coordinates in a general position, and further how to compute the vanishing line via the eigenvector representation. We also propose that the projectivity may be expressed directly and analytically from the vanishing line and three 3D–2D point correspondences. It is shown that plane affine properties may be computed and the metric may be recovered from known metric information, which includes an angle, two equal but unknown angles, and a length ratio of two non-parallel line segments, without using the image of the circular points as an intermediate step. The correctness and performance of the novel results are demonstrated by thorough testing on both synthetic and real data.     

Quasi-Invariant Parameterisations and Matching of Curves in Images

In this paper, we investigate quasi-invariance on a smooth manifold, and show that there exist quasi-invariant parameterisations which are not exactly invariant but approximately invariant under group transformations and do not require high order derivatives. The affine quasi-invariant parameterisation is investigated in more detail and exploited for defining general affine semi-local invariants from second order derivatives only. The new invariants are implemented and used for matching curve segments under general affine motions and extracting symmetry axes of objects with 3D bilateral symmetry.     

Part I: Modeling image curves using invariant 3-D object curvemodels-a path to 3-D recognition and shape estimation from imagecontours

This paper and its companion are concerned with the problems of 3-D object recognition and shape estimation from image curves using a 3-D object curve model that is invariant to affine transformation onto the image space, and a binocular stereo imaging system. The objects of interest here are the ones that have markings (e.g., characters, letters, special drawings and symbols, etc.) on their surfaces. The 3-D curves on the object are modeled as B-splines, which are characterized by a set of parameters (the control points) from which the 3-D curve can be totally generated. The B-splines are invariant under affine transformations. That means that the affine projected object curve onto the image space is a B-spline whose control points are related to the object control points through the affine transformation. Part I deals with issues relating to the curve modeling process. In particular, the authors address the problems of estimating the control points from the data curve, and of deciding on the “best” order B-spline and the “best” number of control points to be used to model the image or object curve(s). A minimum mean-square error (mmse) estimation technique which is invariant to affine transformations is presented as a noniterative, simple, and fast approach for control point estimation. The “best” B-spline is decided upon using a Bayesian selection rule. Finally, we present a matching algorithm that allocates a sample curve to one of p prototype curves when the sample curve is an a priori unknown affine transformation of one of the prototype curves stored in the data base. The approach is tried on a variety of images of real objects     

Segmenting Traffic Scenes from Grey Level and Motion Information

This paper is concerned with an efficient estimation and segmentation of 2D motion from image sequences, with the focus on traffic monitoring applications. In order to reduce the computational load to achieve real-time implementation, the proposed approach makes use of simplifying assumptions that the camera is stationary, and that the projection of vehicles motion on the image plane can be approximated by translation. We show that satisfactory results can be achieved even under such apparently restrictive assumptions. The use of 2D motion analysis and the pre-segmentation stage significantly reduces the computational load, and the region-based motion estimator gives robustness to noise and changes in the illumination conditions.