A New Linear Method for Camera Self-Calibration with Planar Motion

We consider the self-calibration (affine and metric reconstruction) problem from images acquired with a camera with unchanging internal parameters undergoing planar motion. The general self-calibration methods (modulus constraint, Kruppa equations) are known to fail with this camera motion. In this paper we give two novel linear constraints on the coordinates of the plane at infinity in a projective reconstruction for any camera motion. In the planar case, we show that the two constraints are equivalent and easy to compute, giving us a linear version of the quartic modulus constraint. Using this fact, we present a new linear method to solve the self-calibration problem with planar motion of the camera from three or more images. This work was partly supported by project BFM2003-02914 from the Ministerio de Ciencia y Tecnología (Spain). Ferran Espuny received the MSc in Mathematics in 2002 from the Universitat de Barcelona, Spain. He is currently a PhD student and associate professor in the Departament d’àlgebra i Geometria at Universitat de Barcelona, Spain. His research, supervised by Dr. José Ignacio Burgos Gil, is focussed on self-calibration and critical motions for both pinhole and generic camera models.     

Plane-based camera self-calibration by metric rectification of images

Plane-based self-calibration aims at the computation of camera intrinsic parameters from homographies relating multiple views of the same unknown planar scene. This paper proposes a straightforward geometric statement of plane-based self-calibration, through the concept of metric rectification of images. A set of constraints is derived from a decomposition of metric rectification in terms of intrinsic parameters and planar scene orientation. These constraints are then solved using an optimization framework based on the minimization of a geometrically motivated cost function. The link with previous approaches is demonstrated and our method appears to be theoretically equivalent but conceptually simpler. Moreover, a solution dealing with radial distortion is introduced. Experimentally, the method is compared with plane-based calibration and very satisfactory results are obtained. Markerless self-calibration is demonstrated using an intensity-based estimation of the inter-image homographies.     

Local surface shape estimation of 3-D textured surfaces usingGaussian Markov random fields and stereo windows

The problem of extracting the local shape information of a 3-D texture surface from a single 2-D image by tracking the perceived systematic deformations the texture undergoes by virtue of being present on a 3-D surface and by virtue of being imaged is examined. The surfaces of interest are planar and developable surfaces. The textured objects are viewed as originating by laying a rubber planar sheet with a homogeneous parent texture on it onto the objects. The homogeneous planar parent texture is modeled by a stationary Gaussian Markov random field (GMRF). A probability distribution function for the texture data obtained by projecting the planar parent texture under a linear camera model is derived, which is an explicit function of the parent GMRF parameters, the surface shape parameters. and the camera geometry. The surface shape parameter estimation is posed as a maximum likelihood estimation problem. A stereo-windows concept is introduced to obtain a unique and consistent parent texture from the image data that, under appropriate transformations, yields the observed texture in the image. The theory is substantiated by experiments on synthesized as well as real images of textured surfaces     

Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters

In this paper the theoretical and practical feasibility of self-calibration in the presence of varying intrinsic camera parameters is under investigation. The paper's main contribution is to propose a self-calibration method which efficiently deals with all kinds of constraints on the intrinsic camera parameters. Within this framework a practical method is proposed which can retrieve metric reconstruction from image sequences obtained with uncalibrated zooming/focusing cameras. The feasibility of the approach is illustrated on real and synthetic examples. Besides this a theoretical proof is given which shows that the absence of skew in the image plane is sufficient to allow for self-calibration. A counting argument is developed which—depending on the set of constraints—gives the minimum sequence length for self-calibration and a method to detect critical motion sequences is proposed.     

Uncalibrated Motion Capture Exploiting Articulated Structure Constraints

We present an algorithm for 3D reconstruction of dynamic articulated structures, such as humans, from uncalibrated multiple views. The reconstruction exploits constraints associated with a dynamic articulated structure, specifically the conservation over time of length between rotational joints. These constraints admit reconstruction of metric structure from at least two different images in each of two uncalibrated parallel projection cameras. As a by product, the calibration of the cameras can also be computed. The algorithm is based on a stratified approach, starting with affine reconstruction from factorization, followed by rectification to metric structure using the articulated structure constraints. The exploitation of these specific constraints admits reconstruction and self-calibration with fewer feature points and views compared to standard self-calibration. The method is extended to pairs of cameras that are zooming, where calibration of the cameras allows compensation for the changing scale factor in a scaled orthographic camera. Results are presented in the form of stick figures and animated 3D reconstructions using pairs of sequences from broadcast television. The technique shows promise as a means of creating 3D animations of dynamic activities such as sports events.     

未标定单幅结构场景图像的三维重构

提出了一种单视三维重构方法，该方法是利用用户提供图像点及其对应的三维点之间几何信息。由于结构场景是由大量平面构成的，存在大量的平行性、正交性约束，因此该方法主要应用于结构场景的三维重构。首先，相机定标和计算每个平面的度量信息，即先基于3组互相垂直方向的影灭点，对方形像素相机标定，再利用影灭线和圆环点像，对每个平面度量校正；然后考虑每个校正平面的尺度因子和非正交平面间的相对面向，从而将所有校正后的平面缝合起来。采用真实图像进行实验，实验结果表明，该方法简单易用。     

Using Specific Displacements to Analyze Motion without Calibration

Considering the field of un-calibrated image sequences and self-calibration, this paper analyzes the use of specific displacements (such as fixed axis rotation, pure translations,...) or specific sets of camera parameters. This allows to induce affine or metric constraints, which can lead to self-calibration and 3D reconstruction.A uniformed formalism for such models already developed in the literature plus some novel models are developed here. A hierarchy of special situations is described, in order to tailor the most appropriate camera model to either the actual robotic device supporting the camera, or to tailor the fact we only have a reduced set of data available.This visual motion perception module leads to the estimation of a minimal 3D parameterization of the retinal displacement for a monocular visual system without calibration, and leads to self-calibration and 3D dynamic analysis.The implementation of these equations is analyzed and experimented.     

Self-Calibration Under the Cayley Framework

The Cayley framework here is meant to tackle the vision problems under the infinite Cayley transformation (ICT), its main advantage lies in its numerical stability. In this work, the stratified self-calibration under the Cayley framework is investigated. It is well known that the main difficulty of the stratified self-calibration in multiple view geometry is to upgrade a projective reconstruction to an affine one, in other words, to estimate the unknown 3-vector of the plane at infinity, called the normal vector. To our knowledge, without any prior knowledge about the scene or the camera motion, the only available constraint on a moving camera with constant intrinsic parameters is the well-known Modulus Constraint in the literature. Do other kinds of constraints exist? If yes, what they are? How could they be used? In this work, such questions will be systematically investigated under the Cayley framework. Our key contributions include: 1. The original projective expression of the ICT is simplified and a new projective expression is derived to make the upgrade easier from a projective reconstruction to a metric reconstruction. 2. The constraints on the normal vector are systematically investigated. For two views, two constraints on the normal vector are derived; one of them is the well-known modulus constraint, while the other is a new inequality constraint. There are only these two constraints for two views. For three views, besides the constraints for two views, two groups of new constraints are derived and each of them contains three constraints. In other words, there are 12 constraints in total for three views. 3. Based on our projective expression and these constraints, a stratified Cayley algorithm and a total Cayley algorithm are proposed for the metric reconstruction from images. It is experimentally shown that they both improve significantly the numerical stability of the classical algorithms. Compared with the global optimal algorithm under the infinite homography framework, the Cayley algorithms have comparable calibration accuracy, but substantially reduce the computational load.     

A new method for camera stratified self-calibration under circular motion

We consider the stratified self-calibration (affine and metric reconstruction) problem from images acquired with a camera with unchanging internal parameters undergoing circular motion. The general stratified method (modulus constraints) is known to fail with this motion. In this paper we give a novel constraint on the plane at infinity in projective reconstruction for circular motion, the constant inter-frame motion constraint on the plane at infinity between every two adjacent views and a fixed view of the motion sequences, by making use of the facts that in many commercial systems rotation angles are constant. An initial solution can be obtained by using the first three views of the sequence, and Stratified Iterative Particle Swarm Optimization (SIPSO) is proposed to get an accurate and robust solution when more views are at hand. Instead of using the traditional optimization algorithm as the last step to obtain an accurate solution, in this paper, the whole motion sequence information is exploited before computing the camera calibration matrix, this results in a more accurate and robust solution. Once the plane at infinity is identified, the calibration matrices of the camera and a metric reconstruction can be readily obtained. Experiments on both synthetic and real image sequence are given, showing the accuracy and robustness of the new algorithm.     

Camera self-calibration from unknown planar structures enforcingthe multiview constraints between collineations

In this paper, we describe an efficient method to impose the constraints existing between the collineations between images which can be computed from a sequence of views of a planar structure. These constraints are usually not taken into account by multiview techniques in order not to increase the computational complexity of the algorithms. However, imposing the constraints is very useful since it allows a reduction of geometric errors in the reprojected features and provides a consistent set of collineations which can be used for several applications such as mosaicing, reconstruction, and self-calibration. In order to show the validity of our approach, this paper focus on self-calibration from unknown planar structures proposing a method exploiting the consistent set of collineations. Our method can deal with an arbitrary number of views and an arbitrary number of planes and varying camera internal parameters. However, for simplicity, this papers will only discuss the case with one plane in several views. The results obtained with synthetic and real data are very accurate and stable even when using only few images     

Fourier domain representation of planar curves for recognition in multiple views

Recognition of planar shapes is an important problem in computer vision and pattern recognition. The same planar object contour imaged from different cameras or from different viewpoints looks different and their recognition is non-trivial. Traditional shape recognition deals with views of the shapes that differ only by simple rotations, translations, and scaling. However, shapes suffer more serious deformation between two general views and hence recognition approaches designed to handle translations, rotations, and/or scaling would prove to be insufficient. Many algebraic relations between matching primitives in multiple views have been identified recently. In this paper, we explore how shape properties and multiview relations can be combined to recognize planar shapes across multiple views. We propose novel recognition constraints that a planar shape boundary must satisfy in multiple views. The constraints are on the rank of a Fourier-domain measurement matrix computed from the points on the shape boundary. Our method can additionally compute the correspondence between the curve points after a match is established. We demonstrate the applications of these constraints experimentally on a number of synthetic and real images.     

Generic self-calibration of central cameras

We consider the self-calibration problem for a generic imaging model that assigns projection rays to pixels without a parametric mapping. We consider the central variant of this model, which encompasses all camera models with a single effective viewpoint. Self-calibration refers to calibrating a camera’s projection rays, purely from matches between images, i.e. without knowledge about the scene such as using a calibration grid. In order to do this we consider specific camera motions, concretely, pure translations and rotations, although without the knowledge of rotation and translation parameters (rotation angles, axis of rotation, translation vector). Knowledge of the type of motion, together with image matches, gives geometric constraints on the projection rays. We show for example that with translational motions alone, self-calibration can already be performed, but only up to an affine transformation of the set of projection rays. We then propose algorithms for full metric self-calibration, that use rotational and translational motions or just rotational motions.     

强约束条件下环绕式相机标定方法

目的 在计算机视觉和摄影测量领域,经常应用多视角图像对场景进行高精度的三维重建。其中,相机内参数和相机间固定相对关系的高精度标定是关键环节,文章提出一种能够在强约束条件下快速进行相机标定的方法。方法 通过相机间6个相互独立的约束,充分利用系统的几何条件,确定固有关系,再以共线方程为基础推导强约束条件下的平差模型,并应用于自检校光束法平差,开展相邻立体相机的匹配,实现多相机系统的快速标定。结果 最后通过实验,验证了加了强约束条件后,加大了平差的多余观测数,提高了标定精度和鲁棒性。结论 建立了相机标定系统,提出了在强约束条件下快速进行相机标定的方法,展开了人体三维重建研究,并且该方法可推广到多个相机组成的多相机立体量测系统的标定中。     

Image mosaicking for polyhedral scene and in particular singly visible surfaces

Image mosaic construction is about stitching together a number of images about the same scene to construct a single image with a larger field of view. The majority of the previous work was rooted at the use of a single image-to-image mapping termed planar homography for representing the imaged scene. However, the mapping is applicable only to cases where the imaged scene is either a single planar surface, or very distant from the cameras, or imaged under a pure rotation of the camera, and that greatly limits the range of applications of the mosaicking methods. This paper presents a novel mosaicking solution for scenes that are polyhedral (thus consisting of multiple surfaces) and that are pictured possibly in closed range of the camera. The solution has two major advantages. First, it requires only a few correspondences over the entire scene, not correspondences over every surface patch in it to work. Second, it conquers a seemingly impossible task—warping image data of surfaces that are visible in only one of the input images, which we refer to as the singly visible surfaces, to another viewpoint to constitute the mosaic there. We also provide a detail analysis of what determines whether a singly visible surface could be mosaicked or not. Experimental results on real image data are presented to illustrate the performance of the method.     

纹理映射中的平面校正技术研究

为了快速实时地进行由平面组成的结构景物的3D建模问题,文中介绍了一种在进行图像3D重构时纹理映射中的平面校正方法。介绍了三角形模型在图像处理、图形绘制、虚拟现实等技术中的重要作用。从射影几何的角度出发,给出了从两幅视图进行景物三维重构的分层重构方法。在已知欧氏重构即摄像机内参数的基础上,介绍一种基于标定的平面射影失真矫正方法。通过此方法,将矫正过的纹理映射到欧式点重构结构中,得到景物的3D模型。经实验验证,这种方法在处理由平面组成的景物的3D重构中是实时有效的。     

Single view compositing with shadows

In this paper, we describe how geometrically correct and visually realistic shadows may be computed for objects composited into a single view of a target scene. Compared to traditional single view compositing methods, which either do not deal with the shadow effects or manually create the shadows for the composited objects, our approach efficiently utilizes the geometric and photometric constraints extracted from a single target image to synthesize the shadows consistent with the overall target scene for the inserted objects. In particular, we explore (i) the constraints provided by imaged scene structure, e.g. vanishing points of orthogonal directions, for camera calibration and thus explicit determination of the locations of the camera and the light source; (ii) the relatively weaker geometric constraint, the planar homology, that models the imaged shadow relations when explicit camera calibration is not possible; and (iii) the photometric constraints that are required to match the color characteristics of the synthesized shadows with those of the original scene. For each constraint, we demonstrate the working examples followed by our observations. To show the accuracy and the applications of the proposed method, we present the results for a variety of target scenes, including footage from commercial Hollywood movies and 3D video games.     

Visual Modeling with a Hand-Held Camera

In this paper a complete system to build visual models from camera images is presented. The system can deal with uncalibrated image sequences acquired with a hand-held camera. Based on tracked or matched features the relations between multiple views are computed. From this both the structure of the scene and the motion of the camera are retrieved. The ambiguity on the reconstruction is restricted from projective to metric through self-calibration. A flexible multi-view stereo matching scheme is used to obtain a dense estimation of the surface geometry. From the computed data different types of visual models are constructed. Besides the traditional geometry- and image-based approaches, a combined approach with view-dependent geometry and texture is presented. As an application fusion of real and virtual scenes is also shown.     

带有倾斜因子的摄像机自标定方法

摄像机自标定是三维重建技术的基本问题 ,得到许多学者的大力研究 .为了简化摄像机自标定过程 ,一般假设摄像机内参数中的倾斜因子为零 ,然后对主点和焦距进行自标定 .但在摄像机模型为完全的射影模型时 ,即当倾斜因子 (Skew Factor)值较大时 ,则使用上述假设得到的自标定参数误差较大 ,有时甚至无法得到结果 .为了对倾斜因子值较大的摄像机进行准确标定 ,提出了一种当摄像机的倾斜因子已知但不为零时的摄像机自标定方法 ,试验结果证明该方法可以得到比较准确的摄像机内参数 ,并可使得后续的三维重建得到较好的结果 .     

Calibration of multi-beam laser tracking systems

Laser tracking systems (LTSs) employ tracking laser interferometers for coordinate measuring of precision machine tools and robots. Such coordinate measuring machines, if properly calibrated, are potentially fast, very accurate and can cover a large workspace. LTS devices must be self-calibrated, using redundancy or constraint surfaces.A methodology for self-calibration of a multi-beam LTS utilizing planar constraints is formulated and demonstrated. A kinematic model of a multi-beam LTS is derived. Model error analysis demonstrates that the use of angular measurement of the gimbal joint positions, relatively inaccurate as these are, does improve the overall system calibration accuracy. Self-calibration model parameters observability of the multi-beam LTS is studied. The results reveal the applicability of planar constraints to system self-calibration. Simulation and experimentation results obtained on a prototype system are reported demonstrating the applicability of the calibration strategies.     

An improved algorithm for two-image camera self-calibration and Euclidean structure recovery using absolute quadric

We present an improved algorithm for two-image camera self-calibration and Euclidean structure recovery, where the effective focal lengths of both cameras are assumed to be the only unknown intrinsic parameters. By using the absolute quadric, it is shown that the effective focal lengths can be computed linearly from two perspective images without imposing scene or motion constraints. Moreover, a quadratic equation derived from the absolute quadric is proposed for solving the parameters of the plane at infinity from two images, which upgrades a projective reconstruction to a Euclidean reconstruction.