Extension of Affine Shape

In this paper, we extend the notion of affine shape, introduced by Sparr, from finite point sets to more general sets. It turns out to be possible to generalize most of the theory. The extension makes it possible to reconstruct, for example, 3D-curves up to projective transformations, from a number of their 2D-projections. An algorithm is presented, which is independent of choice of coordinates, is robust, does not rely on any preselected parameters and works for an arbitrary number of images. In particular this means that a solution is given to the aperture problem of finding point correspondences between curves.     

Robust Non-rigid Registration Based on Affine ICP Algorithm and Part-Based Method

The classical affine iterative closest point (ICP) algorithm is fast and accurate for affine registration between two point sets, but it is easy to fall into a local minimum. As an extension of the classical affine registration algorithm, this paper first proposes an affine ICP algorithm based on control point guided, and then applies this new method to establish a robust non-rigid registration algorithm based on local affine registration. The algorithm uses a hierarchical iterative method to complete the point set non-rigid registration from coarse to fine. In each iteration, the sub data point sets and sub model point sets are divided, meanwhile, the shape control points of each sub point set are updated. Then we use the control point guided affine ICP algorithm to solve the local affine transformation between the corresponding sub point sets. Next, the local affine transformation obtained by the previous step is used to update the sub data point sets and their shape control point sets. Experimental results demonstrate that the accuracy and convergence of our algorithm are greatly improved compared with the traditional point set non-rigid registration algorithms.     

Part I: Modeling image curves using invariant 3-D object curvemodels-a path to 3-D recognition and shape estimation from imagecontours

This paper and its companion are concerned with the problems of 3-D object recognition and shape estimation from image curves using a 3-D object curve model that is invariant to affine transformation onto the image space, and a binocular stereo imaging system. The objects of interest here are the ones that have markings (e.g., characters, letters, special drawings and symbols, etc.) on their surfaces. The 3-D curves on the object are modeled as B-splines, which are characterized by a set of parameters (the control points) from which the 3-D curve can be totally generated. The B-splines are invariant under affine transformations. That means that the affine projected object curve onto the image space is a B-spline whose control points are related to the object control points through the affine transformation. Part I deals with issues relating to the curve modeling process. In particular, the authors address the problems of estimating the control points from the data curve, and of deciding on the “best” order B-spline and the “best” number of control points to be used to model the image or object curve(s). A minimum mean-square error (mmse) estimation technique which is invariant to affine transformations is presented as a noniterative, simple, and fast approach for control point estimation. The “best” B-spline is decided upon using a Bayesian selection rule. Finally, we present a matching algorithm that allocates a sample curve to one of p prototype curves when the sample curve is an a priori unknown affine transformation of one of the prototype curves stored in the data base. The approach is tried on a variety of images of real objects     

基于Huang-Faugeras约束的相机分步自标定方法

提出一种运动变焦相机分步自标定的方法。因相机在运动过程中其纵横比、主点固定不变，可以事先标定；在假设像素无扭曲的情况下基于Huang-Faugeras约束线性求解焦距，只需输入基础矩阵，不必须进行任务类型的投影阵分解或投影束校正。线性求解焦距避免了非线性法的不稳定性，实际数值实验表明文中方法简单实用。     

基于Hausdorff距离图象配准方法研究

图象配准是图象融合的一个重要步骤，为此提出了一种自动图象配准算法，该算法从两幅待配准的图象中分别抽取特征点，然后选用Hausdorff距离对两特征点集进行匹配，得到点集间的仿射变换，从而实现图象的自动配准，此算法以特征点而不是物体边缘计算仿射变换，大大降低了计算Hausdorff距离的运算量；同时，基于Hausdorff距离的图象匹配只需要点集之间的对应，而无须点与点的对应，因而可以使用于存在较大物体形变的情况，即完成两幅差异较大图象的配准，实验结果证明了算法的有效性。     

Projective Reconstruction from Multiple Views with Minimization of 2D Reprojection Error

The problem of projective reconstruction by minimization of the 2D reprojection error in multiple images is considered. Although bundle adjustment techniques can be used to minimize the 2D reprojection error, these methods being based on nonlinear optimization algorithms require a good starting point. Quasi-linear algorithms with better global convergence properties can be used to generate an initial solution before submitting it to bundle adjustment for refinement. In this paper, we propose a factorization-based method to integrate the initial search as well as the bundle adjustment into a single algorithm consisting of a sequence of weighted least-squares problems, in which a control parameter is initially set to a relaxed state to allow the search of a good initial solution, and subsequently tightened up to force the final solution to approach a minimum point of the 2D reprojection error. The proposed algorithm is guaranteed to converge. Our method readily handles images with missing points.     

Cross-weighted moments and affine invariants for image registrationand matching

A framework for deriving a class of new global affine invariants for both object matching and positioning based on a novel concept of cross-weighted moments with fractional weights is presented. The fractional weight factor allows for a more flexible range to balance between the capability to discriminate between objects that differ only in small shape details and the sensitivity of small shape details to the presence of the noise. Moreover, it makes it possible to arrive at low order (zero order) affine invariants that are more robust than those derived from higher order regular moments. The affine transformation parameters are recovered from the zero and the first order cross-weighted moments without requiring any feature point correspondence information. The equations used to find the affine transformation parameters are linear algebraic. The sensitivity of the cross-weighted moment invariants to noise, missing data, and perspective effects is shown on real images     

Hybrid Parallel Bundle Adjustment for 3D Scene Reconstruction with Massive Points

Bundle adjustment(BA) is a crucial but time consuming step in 3D reconstruction.In this paper,we intend to tackle a special class of BA problems where the reconstructed 3D points are much more numerous than the camera parameters,called Massive-Points BA(MPBA) problems.This is often the case when high-resolution images are used.We present a design and implementation of a new bundle adjustment algorithm for efficiently solving the MPBA problems.The use of hardware parallelism,the multi-core CPUs as well as GPUs,is explored.By careful memory-usage design,the graphic-memory limitation is effectively alleviated.Several modern acceleration strategies for bundle adjustment,such as the mixed-precision arithmetics,the embedded point iteration,and the preconditioned conjugate gradients,are explored and compared.By using several high-resolution image datasets,we generate a variety of MPBA problems,with which the performance of five bundle adjustment algorithms are evaluated.The experimental results show that our algorithm is up to 40 times faster than classical Sparse Bundle Adjustment,while maintaining comparable precision.     

基于仿射点对应的分层重构

提出了一种基于仿射点对应的分层重构方法,所谓仿射点对应是指相差一个仿射变换的两个空间点集的图像对应．该方法主要分为以下三个步骤：首先,从点对应计算准仿射重构;然后,由仿射点对应的准仿射重构建立一个三维射影变换,并利用这个射影变换的特征向量来确定无穷远平面,从而得到仿射重构;最后,从仿射重构所获得的无穷远平面单应矩阵标定摄像机内参数,进而得到度量重构．在上述三个步骤中,第二个步骤是最关键的,即如何确定对应于无穷远平面的特征向量,这也是该文的新思想和主要贡献所在．仿真和真实图像实验均表明,该文的方法是有效的,并且有很好的鲁棒性．     

A computational model of bounded developable surfaces with application to image‐based three‐dimensional reconstruction

Developable surfaces have been extensively studied in computer graphics because they are involved in a large body of applications. This type of surfaces has also been used in computer vision and document processing in the context of three‐dimensional (3D) reconstruction for book digitization and augmented reality. Indeed, the shape of a smoothly deformed piece of paper can be very well modeled by a developable surface. Most of the existing developable surface parameterizations do not handle boundaries or are driven by overly large parameter sets. These two characteristics become issues in the context of developable surface reconstruction from real observations. Our main contribution is a generative model of bounded developable surfaces that solves these two issues. Our model is governed by intuitive parameters whose number depends on the actual deformation and including the “flat shape boundary”. A vast majority of the existing image‐based paper 3D reconstruction methods either require a tightly controlled environment or restricts the set of possible deformations. We propose an algorithm for reconstructing our model's parameters from a general smooth 3D surface interpolating a sparse cloud of 3D points. The latter is assumed to be reconstructed from images of a static piece of paper or any other developable surface. Our 3D reconstruction method is well adapted to the use of keypoint matches over multiple images. In this context, the initial 3D point cloud is reconstructed by structure‐from‐motion for which mature and reliable algorithms now exist and the thin‐plate spline is used as a general smooth surface model. After initialization, our model's parameters are refined with model‐based bundle adjustment. We experimentally validated our model and 3D reconstruction algorithm for shape capture and augmented reality on seven real datasets. The first six datasets consist of multiple images or videos and a sparse set of 3D points obtained by structure‐from‐motion. The last dataset is a dense 3D point cloud acquired by structured light. Our implementation has been made publicly available on the authors' web home pages. Copyright © 2012 John Wiley & Sons, Ltd.     

Non-Rigid Stereo Factorization

In this paper we address the problem of recovering 3D non-rigid structure from a sequence of images taken with a stereo pair. We have extended existing non-rigid factorization algorithms to the stereo camera case and presented an algorithm to decompose the measurement matrix into the motion of the left and right cameras and the 3D shape, represented as a linear combination of basis-shapes. The added constraints in the stereo camera case are that both cameras are viewing the same structure and that the relative orientation between both cameras is fixed. Our focus in this paper is on the recovery of flexible 3D shape rather than on the correspondence problem. We propose a method to compute reliable 3D models of deformable structure from stereo images. Our experiments with real data show that improved reconstructions can be achieved using this method. The algorithm includes a non-linear optimization step that minimizes image reprojection error and imposes the correct structure to the motion matrix by choosing an appropriate parameterization. We show that 3D shape and motion estimates can be successfully disambiguated after bundle adjustment and demonstrate this on synthetic and real image sequences. While this optimization step is proposed for the stereo camera case, it can be readily applied to the case of non-rigid structure recovery using a monocular video sequence. Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.     

结合轮廓与形状特征的仿射形状匹配

目的 针对仿射变换下形状匹配中存在的描述子对形状的描述能力不足,以及描述子计算耗时大的问题,改进基于所有图像点投影的方法,提出一种利用轮廓计算投影面积的仿射形状匹配算法。方法 该算法分为粗匹配和精匹配两个阶段。粗匹配阶段以CSS角点作为备选特征点,首先统计轮廓投影面积分布作为特征点描述子;然后利用动态规划蚁群算法匹配两幅图片公共特征点序列,并将匹配好的特征点序列记为对应的新特征点;最后采用该新特征点划分目标曲线,得到对应的轮廓曲线;这一阶段的目的是对形状的筛选以及寻找一致的轮廓特征点,同时完成轮廓曲线的划分。精匹配阶段,采用小波仿射不变描述子,对粗匹配阶段匹配代价最小的5%的目标进行对应曲线匹配,得到精匹配阶段的匹配代价,从而实现对仿射目标的识别;精匹配弥补了描述子对轮廓细节描述不足的问题。结果 算法的平均检索速度比传统基于形状投影分布描述子提高44.3%,在MPEG-7图像库上的检索效果为98.65%,在MPEG-7仿射图像库上的查准率与查全率综合评价指标比传统的基于形状投影分布描述子高3.1%,比形状上下文高25%。结论 本文算法匹配效果好,效率高,抗噪性强,解决了仿射描述子计算速度慢、描述能力不足的问题,能有效地应用于仿射形状匹配与检索领域。     

Manifold splines with a single extraordinary point

This paper develops a novel computational technique to define and construct manifold splines with only one singular point by employing the rigorous mathematical theory of Ricci flow. The central idea and new computational paradigm of manifold splines are to systematically extend the algorithmic pipeline of spline surface construction from any planar domain to an arbitrary topology. As a result, manifold splines can unify planar spline representations as their special cases. Despite its earlier success, the existing manifold spline framework is plagued by the topology-dependent, large number of singular points (i.e., |2g−2| for any genus-g surface), where the analysis of surface behaviors such as continuity remains extremely difficult. The unique theoretical contribution of this paper is that we devise new mathematical tools so that manifold splines can now be constructed with only one singular point, reaching their theoretic lower bound of singularity for real-world applications. Our new algorithm is founded upon the concept of discrete Ricci flow and associated techniques. First, Ricci flow is employed to compute a special metric of any manifold domain (serving as a parametric domain for manifold splines), such that the metric becomes flat everywhere except at one point. Then, the metric naturally induces an affine atlas covering the entire manifold except this singular point. Finally, manifold splines are defined over this affine atlas. The Ricci flow method is theoretically sound, and practically simple and efficient. We conduct various shape experiments and our new theoretical and algorithmic results alleviate the modeling difficulty of manifold splines, and hence, promote the widespread use of manifold splines in surface and solid modeling, geometric design, and reverse engineering.     

Depth recovery and affine reconstruction under camera pure translation

This paper addresses the problems of depth recovery and affine reconstruction from two perspective images, which are generated by an uncalibrated translating camera. Firstly, we develop a new constraint that the homography for the plane, which is orthogonal to the optical axis, is determined only by the epipole and the plane's relative distance to the origin under camera pure translation. The algorithm of depth recovery is based on this new constraint, and it can successfully avoid the step of camera calibration. With the recovered depth, we show that affine reconstruction can be obtained readily. The proposed affine reconstruction does not need any control points, which were used to expand the affine coordinate system in existing method. Therefore, it could avoid the step of non-planarity verification as well as the errors from the control points. Error analysis is also presented to evaluate the uncertainty for the recovered depth value. Finally, we have tested the proposed algorithm with both simulated data and real image data. And the results show that the proposed algorithm is accurate and practical.     

Efficient planar object tracking and parameter estimation usingcompactly represented cubic B-spline curves

In this paper, we consider the problem of matching 2D planar object curves from a database, and tracking moving object curves through an image sequence. The first part of the paper describes a curve data compression method using B-spline curve approximation. We present a new constrained active B-spline curve model based on the minimum mean square error (MMSE) criterion, and an iterative algorithm for selecting the “best” segment border points for each B-spline curve. The second part of the paper describes a method for simultaneous object tracking and affine parameter estimation using the approximate curves and profiles. We propose a novel B-spline point assignment algorithm which incorporates the significant corners for interpolating corresponding points on the two curves to be compared. A gradient-based algorithm is presented for simultaneously tracking object curves, and estimating the associated translation, rotation and scaling parameters. The performance of each proposed method is evaluated using still images and image sequences containing simple objects     

Drift detection and removal for sequential structure from motion algorithms

In sequential Structure from Motion algorithms for extended image or video sequences, error build up caused by drift poses a problem as feature tracks that normally represent a single scene point will have distinct 3D reconstructions. For the final bundle adjustment to remove this drift, it must be told about these 3D-3D correspondences through a change in the cost function. However, as a bundle adjustment is a nonlinear optimization technique, the drift needs to be removed from the supplied initial solution to allow for convergence of the bundle adjustment to the real global optimum. Before drift can be removed, it has to be detected. This is accomplished through understanding of the long term behavior of drift which leaves 3D reconstructions from short sequences intact. Drift detection boils down to identifying reconstructions of the same scene part that only differ up to a projective transformation. After detection, the drift can be removed from future processed images and an Adapted Bundle Adjustment using correspondences supplied by the drift detection can remove the drift from previous images. Several experiments on real video sequences demonstrate the merit of drift detection and removal.