Simplified Active Calibration

We present a new mathematical formulation to estimate the intrinsic parameters of a camera in active or robotic platforms. We show that the focal lengths can be estimated using only one point correspondence that relates images taken before and after a degenerate rotation of the camera. The estimated focal lengths are then treated as known parameters to obtain a linear set of equations to calculate the principal point. Assuming that the principal point is close to the image center, the accuracy of the linear equations are increased by integrating the image center into the formulation. We extensively evaluate the formulations on a simulated camera, 3D scenes and real-world images. Our error analysis over simulated and real images indicates that the proposed Simplified Active Calibration method estimates the parameters of a camera with low error rates that can be used as an initial guess for further non-linear refinement procedures. Simplified Active Calibration can be employed in real-time environments for automatic calibrations given the proposed closed-form solutions.     

Optimizing PTZ camera calibration from two images

In this paper, we address the problem of calibrating an active pan–tilt–zoom (PTZ) camera. In this regard, we make three main contributions: first, for the general camera rotation, we provide a novel solution that yields four independent constraints from only two images, by directly decomposing the infinite homography using a series of Givens rotations. Second, for a camera varying its focal length, we present a solution for the degenerate cases of pure pan and pure tilt that occur very frequently in practical applications of PTZ cameras. Third, we derive a new optimized error function for pure rotation or pan–tilt rotation, which plays a similar role as the epipolar constraint in a freely moving camera, in terms of characterizing the reprojection error of point correspondences. Our solutions and analysis are thoroughly validated and tested on both synthetic and real data, whereby the new geometric error function is shown to outperform existing methods in terms of accuracy and noise resilience.     

Inside looking out camera pose estimation for virtual studio

This paper studies the inside looking out camera pose estimation for the virtual studio. The camera pose estimation process, the process of estimating a camera’s extrinsic parameters, is based on closed-form geometrical approaches which use the benefit of simple corner detection of 3D cubic-like virtual studio landmarks. We first look at the effective parameters of the camera pose estimation process for the virtual studio. Our studies include all characteristic landmark parameters like landmark lengths, landmark corner angles and their installation position errors and some camera parameters like lens focal length and CCD resolution. Through computer simulation we investigate and analyze all these parameters’ efficiency in camera extrinsic parameters, including camera rotation and position matrixes. Based on this work, we found that the camera translation vector is affected more than other camera extrinsic parameters because of the noise of effective camera pose estimation parameters. Therefore, we present a novel iterative geometrical noise cancellation method for the closed-form camera pose estimation process. This is based on the collinearity theory that reduces the estimation error of the camera translation vector, which plays a major role in camera extrinsic parameters estimation errors. To validate our method, we test it in a complete virtual studio simulation. Our simulation results show that they are in the same order as those of some commercial systems, such as the BBC and InterSense IS-1200 VisTracker.     

Quaternion‐based visual servo control in the presence of camera calibration error

Visual servo control systems use information from images along with knowledge of the optic parameters (i.e. camera calibration) to position the camera relative to some viewed object. If there are inaccuracies in the camera calibration, then performance degradation and potentially unpredictable response from the visual servo control system may occur. Motivated by the desire to incorporate robustness to the camera calibration, different control methods have been developed. Previous adaptive/robust controllers (especially for six degree‐of‐freedom camera motion) rely heavily on properties of the rotation parameterization to formulate state estimates and a measurable closed‐loop error system. All of these results are based on the singular axis–angle parameterization. Motivated by the desire to express the rotation by a non‐singular parameterization, efforts in this paper address the question: Can state estimates and a measurable closed‐loop error system be crafted in terms of the quaternion parameterization when the camera calibration parameters are unknown? To answer this question, a contribution of this paper is the development of a robust controller and closed‐loop error system based on a new quaternion‐based estimate of the rotation error. A Lyapunov‐based analysis is provided which indicates that the controller yields asymptotic regulation of the rotation and translation error signals given a sufficient approximate of the camera calibration parameters. Simulation results are provided that illustrate the performance of the controller for a range of calibration uncertainty. Copyright © 2009 John Wiley & Sons, Ltd.     

摄像机旋转运动下的多帧图像对齐技术

针对传统旋转运动参数估计都是采用两帧图像对齐技术,提出了为多帧运动参数估计方法,即使用多帧子空间约束技术．证明了当摄像机参数不变时,多帧运动参数集合可嵌入一个低维线性子空间上;使用奇异值分解方法来降低线性子空间的秩,用最小二乘技术求解所有帧的运动参数．该方法不需要恢复任何3D信息;由于多帧参数估计法比两帧有更多的约束,因此取得更精确的图像对齐效果．该方法可用小图像进行参数估计．     

A new formulation of visual servoing based on cylindrical coordinate system

Image-based visual servoing is a flexible and robust technique to control a robot and guide it to a desired position only by using two-dimensional visual data. However, it is well known that the classical visual servoing based on the Cartesian coordinate system has one crucial problem, that the camera moves backward at infinity, in case that the camera motion from the initial to desired poses is a pure rotation of 1800 around the optical axis. This paper proposes a new formulation of visual servoing, based on a cylindrical coordinate system that can shift the position of the origin. The proposed approach can interpret from a pure rotation around an arbitrary axis to the proper camera rotational motion. It is shown that this formulation contains the classical approach based on the Cartesian coordinate system as an extreme case with the origin located at infinity. Furthermore, we propose a decision method of the origin-shift parameters by estimating a rotational motion from the differences between initial and desired image-plane positions of feature points.     

Towards robust metric reconstruction via a dynamic uncalibrated stereo head

We consider the problem of metrically reconstructing a scene viewed by a moving stereo head. The head comprises two cameras with coplanar optical axes arranged on a lateral rig, each camera being free to vary its angle of vergence. Under various constraints, we derive novel explicit forms for the epipolar equation, and show that a static stereo head constitutes a degenerate camera configuration for carrying out self-calibration. The situation is retrieved by consideration of a stereo head undergoing ground plane motion, and new closed-form solutions for self-calibration are derived. An error analysis reveals that reconstruction is adversely affected by inward-facing camera vergence angles that are similar in value, and by a principal point location whose horizontal component is in error. It is also shown that the adoption of domain-specific robust techniques for computation of the fundamental matrix can significantly improve the quality of scene reconstruction. Experiments conducted with dynamic stereo head images confirm that avoidance of near-degenerate configurations and use of robustness techniques are essential if reliable reconstructions are to be attained.     

Multi-stage 3D reconstruction under circular motion

In this paper, we propose a new method for 3D reconstruction from an image sequence captured by a camera with constant intrinsic parameters undergoing circular motion. We introduce a method, called circular projective reconstruction, for enforcing the circular constraint in a factorization-based projective reconstruction. To deal with the missing data problem, our method uses a multi-stage approach to reconstructing the objects and cameras, which first computes a circular projective reconstruction of a sub-sequence and then extends the reconstruction to the complete sequence. Camera matrix, rotation angles, and 3D structure are computed iteratively in a way that the 2D reprojection error is minimized. The algorithm is evaluated using real image sequences.     

Multiview constraints on homographies

The image motion of a planar surface between two camera views is captured by a homography (a 2D projective transformation). The homography depends on the intrinsic and extrinsic camera parameters, as well as on the 3D plane parameters. While camera parameters vary across different views, the plane geometry remains the same. Based on this fact, we derive linear subspace constraints on the relative homographies of multiple (⩾ 2) planes across multiple views. The paper has three main contributions: 1) We show that the collection of all relative homographies (homologies) of a pair of planes across multiple views, spans a 4-dimensional linear subspace. 2) We show how this constraint can be extended to the case of multiple planes across multiple views. 3) We show that, for some restricted cases of camera motion, linear subspace constraints apply also to the set of homographies of a single plane across multiple views. All the results derived are true for uncalibrated cameras. The possible utility of these multiview constraints for improving homography estimation and for detecting nonrigid motions are also discussed     

未标定摄像机P5P问题的一种解析解

经典PnP问题是以摄像机内参已知为前提条件的,然而对未标定摄像机PnP问题的研究更具有实际意义.文中对未标定四参数针孔摄像机P5P问题的解析解进行了研究,不仅可以求出摄像机相对于世界坐标系的位姿,而且还能得到摄像机的内参.首先根据投影方程和旋转矩阵的性质,利用16个变量构造出了16个约束方程,然后通过消元推导出只含一个未知数的4次多项式方程,分析证实一般情况下未标定摄像机P5P问题最多有4组解.大量的仿真实验表明该算法在确定摄像机位姿上精度很高,且鲁棒性很强.该算法在物体定位、手眼定标、路标导航等领域具有比较重要的实际应用价值.     

Using vanishing points for camera calibration

In this article a new method for the calibration of a vision system which consists of two (or more) cameras is presented. The proposed method, which uses simple properties of vanishing points, is divided into two steps. In the first step, the intrinsic parameters of each camera, that is, the focal length and the location of the intersection between the optical axis and the image plane, are recovered from a single image of a cube. In the second step, the extrinsic parameters of a pair of cameras, that is, the rotation matrix and the translation vector which describe the rigid motion between the coordinate systems fixed in the two cameras are estimated from an image stereo pair of a suitable planar pattern. Firstly, by matching the corresponding vanishing points in the two images the rotation matrix can be computed, then the translation vector is estimated by means of a simple triangulation. The robustness of the method against noise is discussed, and the conditions for optimal estimation of the rotation matrix are derived. Extensive experimentation shows that the precision that can be achieved with the proposed method is sufficient to efficiently perform machine vision tasks that require camera calibration, like depth from stereo and motion from image sequence.     

Multiple motion scene reconstruction with uncalibrated cameras

In this paper, we describe a reconstruction method for multiple motion scenes, which are scenes containing multiple moving objects, from uncalibrated views. Assuming that the objects are moving with constant velocities, the method recovers the scene structure, the trajectories of the moving objects, the camera motion, and the camera intrinsic parameters (except skews) simultaneously. We focus on the case where the cameras have unknown and varying focal lengths while the other intrinsic parameters are known. The number of the moving objects is automatically detected without prior motion segmentation. The method is based on a unified geometrical representation of the static scene and the moving objects. It first performs a projective reconstruction using a bilinear factorization algorithm and, then, converts the projective solution to a Euclidean one by enforcing metric constraints. Experimental results on synthetic and real images are presented.     

摄像机自标定的线性理论与算法

文中提出一种新的摄像机线性自标定的算法和理论。与文献中已有的方法相比,该文方法的主要优点是对摄像机的运动要求不苛刻,如不要求摄像机的运动为正交运动。该方法的关键步骤是确定无穷远平面的单应性矩阵（Homography)。文中从理论上严格证明了下述结论：摄像机作两组运动参数未知的运动M1＝{（R1,t^11),(R1,t^12)},M2={(R2,t^21),(R2,t^22)},若下述两个条件满足：（1）T1＝{t^11,t^12},T2={t^21,t^22}是两个线性无关组（即本组内的两个平移向量线性无关）;（2）R1,R2的旋转轴不同,则可线性地唯一确定摄像机的内参矩阵和运动参数。另外,在四参数摄像机模型下,严格证明了一组运动可线性地唯一确定摄像机的内参数矩阵和运动参数。模拟实验和实际图像实验验证了本文方法的正确性和可行性。     

A Direct Closed-Form Solution to General Relative Orientation of Two Stereo Views

Recovering the relative geometry of two perspective stereo images is a classical problem in photogrammetry and also a basic problem in computer vision. General relative orientation refers to recovering five relative orientation parameters and two principal distances from pure image measurements. This paper presents a new direct closed-form solution to this problem. The approach recasts the explicit stereo coplanarity equation into an implicit form whose coefficients are solvable via a direct solution from the image measurements. Two principal distances, the relative baseline vector and rotation of two images are then solved successively in closed-form from these coefficients. This direct closed-form solution is useful not only in real-time implementation of stereo vision, but also for providing the approximate values of unknown geometric parameters to high-precision photogrammetry, as well as for direct functional analysis of error propagation.     

The Number of Independent Kruppa Constraints from <Emphasis Type="Italic">N</Emphasis> Images

It is well known that without any priori knowledge on the scene, camera motion and camera intrinsic parameters, the only constraint between a pair of images is the so-called epipolar constraint, or equivalently its fundamental matrix. For each fundamental matrix, at most two independent constraints on the cameras＇ intrinsic parameters are available via the Kruppa equations. Given N images, N（N- 1）/2 fundamental matrices appear, and N（N- 1） Kruppa constraints are available. However, to our knowledge, a formal proof of how many independent Kruppa constraints exist out of these N（N - 1） ones is unavailable in the literature. In this paper, we prove that given N images captured by a pinhole camera with varying parameters and under general motion, the number of independent Kruppa constraints is （5N - 9） （N 〉 2）, and it is less than that of independent constraints from the absolute quadric by only one. This one-constraint-less property of the Kruppa equations is their inherent deficiency and is independent of camera motion. This deficiency is due to their failure of automatic enforcement of the rank-three-ness on the absolute quadric.     

基于环视图象的数字相机内参数自定标方法

描述了一种适用于IBR系统的数字相机内参数自定标方法。该方法基于跟踪机机旋转得到的图象系列的特征匹配点以,而不需要标定物。认定在相机旋转过程中,其光学中心是稳定不变的,也即图象中心是固定的,可以事先定标;但容许相机的焦距在各幅图象间有变化,利用真实图象序列进行了实验验证,表明该方法能鲁棒地估算相机内参数。     

Intrinsic and extrinsic active self-calibration of multi-camera systems

We present a method for active self-calibration of multi-camera systems consisting of pan-tilt zoom cameras. The main focus of this work is on extrinsic self-calibration using active camera control. Our novel probabilistic approach avoids multi-image point correspondences as far as possible. This allows an implicit treatment of ambiguities. The relative poses are optimized by actively rotating and zooming each camera pair in a way that significantly simplifies the problem of extracting correct point correspondences. In a final step we calibrate the entire system using a minimal number of relative poses. The selection of relative poses is based on their uncertainty. We exploit active camera control to estimate consistent translation scales for triplets of cameras. This allows us to estimate missing relative poses in the camera triplets. In addition to this active extrinsic self-calibration we present an extended method for the rotational intrinsic self-calibration of a camera that exploits the rotation knowledge provided by the camera’s pan-tilt unit to robustly estimate the intrinsic camera parameters for different zoom steps as well as the rotation between pan-tilt unit and camera. Quantitative experiments on real data demonstrate the robustness and high accuracy of our approach. We achieve a median reprojection error of $0.95$ pixel.     

Trajectory reconstruction with uncertainty estimation using mosaic registration

This paper addresses the problem of estimating the 3D trajectory and associated uncertainty of an underwater autonomous vehicle from a set of images of the seabed taken by an onboard camera. The presented algorithms resort to the use of video mosaics and build upon previous work on image registration and visual pose estimation. The pose estimation is accomplished in two steps. Firstly, a video mosaic is created automatically, covering a region of interest of the seabed. Then, after associating a 3D referential for the mosaic, the estimation of the camera position from a new view of the scene becomes possible.

The main contribution of this paper lies on the assessment of the performance of the 3D pose algorithms. In order to do this, an image sequence with available ground-truth is used for precise error measuring. A first-order error propagation analysis is presented, relating the uncertainty in the location of the match points with the uncertainty in the pose parameters. The importance of predicting the estimate uncertainty is emphasized by the fact that it can be used for comparing algorithms and for the on-line monitoring of the vehicle trajectory reconstruction quality.

Several iterative and non-iterative pose estimation methods are discussed, differing both on the criteria being minimized and on the required information about the camera intrinsic parameters. This information ranges from the full knowledge of the parameters, to the case where they are estimated using self-calibration from an image sequence under pure rotation. The implemented pose algorithms are compared for the accuracy and estimate covariance.     

Stereo calibration from rigid motions

We describe a method for calibrating a stereo pair of cameras using general or planar motions. The method consists of upgrading a 3D projective representation to affine and to Euclidean without any knowledge, neither about the motion parameters nor about the 3D layout. We investigate the algebraic properties relating projective representation to the plane at infinity and to the intrinsic camera parameters when the camera pair is considered as a moving rigid body. We show that all the computations can be carried out using standard linear resolutions techniques. An error analysis reveals the relative importance of the various steps of the calibration process: projective-to-affine and affine-to-metric upgrades. Extensive experiments performed with calibrated and natural data confirm the error analysis as well as the sensitivity study performed with simulated data     

Generic self-calibration of central cameras

We consider the self-calibration problem for a generic imaging model that assigns projection rays to pixels without a parametric mapping. We consider the central variant of this model, which encompasses all camera models with a single effective viewpoint. Self-calibration refers to calibrating a camera’s projection rays, purely from matches between images, i.e. without knowledge about the scene such as using a calibration grid. In order to do this we consider specific camera motions, concretely, pure translations and rotations, although without the knowledge of rotation and translation parameters (rotation angles, axis of rotation, translation vector). Knowledge of the type of motion, together with image matches, gives geometric constraints on the projection rays. We show for example that with translational motions alone, self-calibration can already be performed, but only up to an affine transformation of the set of projection rays. We then propose algorithms for full metric self-calibration, that use rotational and translational motions or just rotational motions.