Identical Projective Geometric Properties of Central Catadioptric Line Images and Sphere Images with Applications to Calibration

Central catadioptric cameras are imaging devices that use mirrors to enhance the field of view while preserving a single effective viewpoint. Lines and spheres in space are all projected into conics in the central catadioptric image planes, and such conics are called line images and sphere images, respectively. We discovered that there exists an imaginary conic in the central catadioptric image planes, defined as the modified image of the absolute conic (MIAC), and by utilizing the MIAC, the novel identical projective geometric properties of line images and sphere images may be exploited: Each line image or each sphere image is double-contact with the MIAC, which is an analogy of the discovery in pinhole camera that the image of the absolute conic (IAC) is double-contact with sphere images. Note that the IAC also exists in the central catadioptric image plane, but it does not have the double-contact properties with line images or sphere images. This is the main reason to propose the MIAC. From these geometric properties with the MIAC, two linear calibration methods for central catadioptric cameras using sphere images as well as using line images are proposed in the same framework. Note that there are many linear approaches to central catadioptric camera calibration using line images. It seems that to use the properties that line images are tangent to the MIAC only leads to an alternative geometric construction for calibration. However, for sphere images, there are only some nonlinear calibration methods in literature. Therefore, to propose linear methods for sphere images may be the main contribution of this paper. Our new algorithms have been tested in extensive experiments with respect to noise sensitivity.     

Geometric properties of central catadioptric line images and their application in calibration

In central catadioptric systems lines in a scene are projected to conic curves in the image. This work studies the geometry of the central catadioptric projection of lines and its use in calibration. It is shown that the conic curves where the lines are mapped possess several projective invariant properties. From these properties, it follows that any central catadioptric system can be fully calibrated from an image of three or more lines. The image of the absolute conic, the relative pose between the camera and the mirror, and the shape of the reflective surface can be recovered using a geometric construction based on the conic loci where the lines are projected. This result is valid for any central catadioptric system and generalizes previous results for paracatadioptric sensors. Moreover, it is proven that systems with a hyperbolic/elliptical mirror can be calibrated from the image of two lines. If both the shape and the pose of the mirror are known, then two line images are enough to determine the image of the absolute conic encoding the camera's intrinsic parameters. The sensitivity to errors is evaluated and the approach is used to calibrate a real camera.     

Catadioptric Projective Geometry

Catadioptric sensors are devices which utilize mirrors and lenses to form a projection onto the image plane of a camera. Central catadioptric sensors are the class of these devices having a single effective viewpoint. In this paper, we propose a unifying model for the projective geometry induced by these devices and we study its properties as well as its practical implications. We show that a central catadioptric projection is equivalent to a two-step mapping via the sphere. The second step is equivalent to a stereographic projection in the case of parabolic mirrors. Conventional lens-based perspective cameras are also central catadioptric devices with a virtual planar mirror and are, thus, covered by the unifying model. We prove that for each catadioptric projection there exists a dual catadioptric projection based on the duality between points and line images (conics). It turns out that planar and parabolic mirrors build a dual catadioptric projection pair. As a practical example we describe a procedure to estimate focal length and image center from a single view of lines in arbitrary position for a parabolic catadioptric system.     

Catadioptric camera calibration using geometric invariants

Central catadioptric cameras are imaging devices that use mirrors to enhance the field of view while preserving a single effective viewpoint. In this paper, we propose a novel method for the calibration of central catadioptric cameras using geometric invariants. Lines and spheres in space are all projected into conics in the catadioptric image plane. We prove that the projection of a line can provide three invariants whereas the projection of a sphere can only provide two. From these invariants, constraint equations for the intrinsic parameters of catadioptric camera are derived. Therefore, there are two kinds of variants of this novel method. The first one uses projections of lines and the second one uses projections of spheres. In general, the projections of two lines or three spheres are sufficient to achieve catadioptric camera calibration. One important conclusion in this paper is that the method based on projections of spheres is more robust and has higher accuracy than that based on projections of lines. The performances of our method are demonstrated by both the results of simulations and experiments with real images.     

一种反射折射摄像机的简易标定方法

Central catadioptric cameras are widely used in virtual reality and robot navigation,and the camera calibration is a prerequisite for these applications.In this paper,we propose an easy calibration method for central catadioptric cameras with a 2D calibration pattern.Firstly,the bounding ellipse of the catadioptric image and field of view (FOV) are used to obtain the initial estimation of the intrinsic parameters.Then,the explicit relationship between the central catadioptric and the pinhole model is used to initialize the extrinsic parameters.Finally,the intrinsic and extrinsic parameters are refined by nonlinear optimization.The proposed method does not need any fitting of partial visible conic,and the projected images of 2D calibration pattern can easily cover the whole image,so our method is easy and robust.Experiments with simulated data as well as real images show the satisfactory performance of our proposed calibration method.     

一种反射折射摄像机的简易标定方法

Central catadioptric cameras are widely used in virtual reality and robot navigation, and the camera calibration is a prerequisite for these applications. In this paper, we propose an easy calibration method for central catadioptric cameras with a 2D calibration pattern. Firstly, the bounding ellipse of the catadioptric image and field of view (FOV) are used to obtain the initial estimation of the intrinsic parameters. Then, the explicit relationship between the central catadioptric and the pinhole model is used to initialize the extrinsic parameters. Finally, the intrinsic and extrinsic parameters are refined by nonlinear optimization. The proposed method does not need any fitting of partial visible conic, and the projected images of 2D calibration pattern can easily cover the whole image, so our method is easy and robust. Experiments with simulated data as well as real images show the satisfactory performance of our proposed calibration method.     

A Geometric Algebra Model for the Image Formation Process of Paracatadioptric Cameras

Omnidirectional vision sensors capture a wide field of view than can benefit many robotic applications. One type of omnidirectional vision sensor is the paracatadioptric. A paracatadioptric sensor combines a parabolic mirror and a camera inducing an orthographic projection. This combination provides a wide field of view while maintaining the single center of projection which is a desirable property of these sensors. Furthermore, lines are projected as circles on the paracatadioptric image plane. In contrast with traditional perspective cameras the image formation process of paracatadioptric sensors is no longer linear. However in this paper we present a model which is able to linearize it. This linearization is based on the fact that the paracatadioptric projection can be represented by a sphere inversion, that belongs to the conformal group Rⁿ\mathcal{R}^{n} which is isomorphic to the Lorentz group in Rⁿ⁺¹\mathcal{R}^{n+1}. Thus a nonlinear conformal transformation can be represented with an equivalent linear Lorenz transformation, which can be represented as a versor in the CGA. Therefore the present model can be applied algebraically not only to points, but also to point-pairs, lines, circles in the same way to all them and in a linear form. The benefits of the proposed method will be reflected on the development of complex applications that use paracatadioptric sensors.     

Hypercatadioptric line images for 3D orientation and image rectification

In central catadioptric systems 3D lines are projected into conics. In this paper we present a new approach to extract conics in the raw catadioptric image, which correspond to projected straight lines in the scene. Using the internal calibration and two image points we are able to compute analytically these conics which we name hypercatadioptric line images. We obtain the error propagation from the image points to the 3D line projection in function of the calibration parameters. We also perform an exhaustive analysis on the elements that can affect the conic extraction accuracy. Besides that, we exploit the presence of parallel lines in man-made environments to compute the dominant vanishing points (VPs) in the omnidirectional image. In order to obtain the intersection of two of these conics we analyze the self-polar triangle common to this pair. With the information contained in the vanishing points we are able to obtain the 3D orientation of the catadioptric system. This method can be used either in a vertical stabilization system required by autonomous navigation or to rectify images required in applications where the vertical orientation of the catadioptric system is assumed. We use synthetic and real images to test the proposed method. We evaluate the 3D orientation accuracy with a ground truth given by a goniometer and with an inertial measurement unit (IMU). We also test our approach performing vertical and full rectifications in sequences of real images.     

A Fisher-Rao Metric for Paracatadioptric Images of Lines

In a central paracatadioptric imaging system a perspective camera takes an image of a scene reflected in a paraboloidal mirror. A 360° field of view is obtained, but the image is severely distorted. In particular, straight lines in the scene project to circles in the image. These distortions make it difficult to detect projected lines using standard image processing algorithms. The distortions are removed using a Fisher-Rao metric which is defined on the space of projected lines in the paracatadioptric image. The space of projected lines is divided into subsets such that on each subset the Fisher-Rao metric is closely approximated by the Euclidean metric. Each subset is sampled at the vertices of a square grid and values are assigned to the sampled points using an adaptation of the trace transform. The result is a set of digital images to which standard image processing algorithms can be applied. The effectiveness of this approach to line detection is illustrated using two algorithms, both of which are based on the Sobel edge operator. The task of line detection is reduced to the task of finding isolated peaks in a Sobel image. An experimental comparison is made between these two algorithms and third algorithm taken from the literature and based on the Hough transform.     

Fitting conics to paracatadioptric projections of lines

The paracatadioptric camera is one of the most popular panoramic systems currently available in the market. It provides a wide field of view by combining a parabolic shaped mirror with a camera inducing an orthographic projection. Previous work proved that the paracatadioptric projection of a line is a conic curve, and that the sensor can be fully calibrated from the image of three or more lines. However, the estimation of the conic curves where the lines are projected is hard to accomplish because of the partial occlusion. In general only a small arc of the conic is visible in the image, and conventional conic fitting techniques are unable to accurately estimate the curve. The present work provides methods to overcome this problem. We show that in uncalibrated paracatadioptric views a set of conic curves is a set of line projections if and only if certain properties are verified. These properties are used to constrain the search space and correctly estimate the curves. The conic fitting is solved naturally by an eigensystem whenever the camera is skewless and the aspect ratio is known. For the general situation the line projections are estimated using non-linear optimization. The set of paracatadioptric lines is used in a geometric construction to determine the camera parameters and calibrate the system. We also propose an algorithm to estimate the conic locus corresponding to a line projection in a calibrated paracatadioptric image. It is proved that the set of all line projections is a hyperplane in the space of conic curves. Since the position of the hyperplane depends only on the sensor parameters, the accuracy of the estimation can be improved by constraining the search to conics lying in this subspace. We show that the fitting problem can be solved by an eigensystem, which leads to a robust and computationally efficient method for paracatadioptric line estimation.     

Calibration of Central Catadioptric Cameras Using a DLT-Like Approach

In this study, we present a calibration technique that is valid for all single-viewpoint catadioptric cameras. We are able to represent the projection of 3D points on a catadioptric image linearly with a 6×10 projection matrix, which uses lifted coordinates for image and 3D points. This projection matrix can be computed from 3D–2D correspondences (minimum 20 points distributed in three different planes). We show how to decompose it to obtain intrinsic and extrinsic parameters. Moreover, we use this parameter estimation followed by a non-linear optimization to calibrate various types of cameras. Our results are based on the sphere camera model which considers that every central catadioptric system can be modeled using two projections, one from 3D points to a unitary sphere and then a perspective projection from the sphere to the image plane. We test our method both with simulations and real images, and we analyze the results performing a 3D reconstruction from two omnidirectional images.     

Self-calibration of hybrid central catadioptric and perspective cameras

Hybrid central catadioptric and perspective cameras are desired in practice, because the hybrid camera system can capture large field of view as well as high-resolution images. However, the calibration of the system is challenging due to heavy distortions in catadioptric cameras. In addition, previous calibration methods are only suitable for the camera system consisting of perspective cameras and catadioptric cameras with only parabolic mirrors, in which priors about the intrinsic parameters of perspective cameras are required. In this work, we provide a new approach to handle the problems. We show that if the hybrid camera system consists of at least two central catadioptric and one perspective cameras, both the intrinsic and extrinsic parameters of the system can be calibrated linearly without priors about intrinsic parameters of the perspective cameras, and the supported central catadioptric cameras of our method can be more generic. In this work, an approximated polynomial model is derived and used for rectification of catadioptric image. Firstly, with the epipolar geometry between the perspective and rectified catadioptric images, the distortion parameters of the polynomial model can be estimated linearly. Then a new method is proposed to estimate the intrinsic parameters of a central catadioptric camera with the parameters in the polynomial model, and hence the catadioptric cameras can be calibrated. Finally, a linear self-calibration method for the hybrid system is given with the calibrated catadioptric cameras. The main advantage of our method is that it cannot only calibrate both the intrinsic and extrinsic parameters of the hybrid camera system, but also simplify a traditional nonlinear self-calibration of perspective cameras to a linear process. Experiments show that our proposed method is robust and reliable.     

Calibrating effective focal length for central catadioptric cameras using one space line

In camera calibration, focal length is the most important parameter to be estimated, while other parameters can be obtained by prior information about scene or system configuration. In this paper, we present a polynomial constraint on the effective focal length with the condition that all the other parameters are known. The polynomial degree is 4 for paracatadioptric cameras and 16 for other catadioptric cameras. However, if the skew is 0 or the ratio between the skew and effective focal length is known, the constraint becomes a linear one or a polynomial one with degree 4 on the effective focal length square for paracatadioptric cameras and other catadioptric cameras, respectively. Based on this constraint, we propose a simple method for estimation of the effective focal length of central catadioptric cameras. Unlike many approaches using lines in literature, the proposed method needs no conic fitting of line images, which is error-prone and highly affects the calibration accuracy. It is easy to implement, and only a single view of one space line is enough with no other space information needed. Experiments on simulated and real data show this method is robust and effective.     

A unifying geometric representation for central projection systems

In this paper, we study projection systems with a single effective viewpoint, including combinations of mirrors and lenses (catadioptric) as well as just lenses with or without radial distortion (dioptric systems). First, we extend a well-known unifying model for central catadioptric systems to incorporate a class of dioptric systems with radial distortion. Second, we provide a new representation for the image plane of central systems. This representation is the lifting through a Veronese map of the original image plane to the 5D projective space. We study how a collineation in the original image plane can be transferred to a collineation in the lifted space, and we prove that in the case of central parabolic systems and cameras with lens distortion the locus of the lifted points representing projections of world lines is a plane. The similarities between paracatadioptric systems and lens with radial distortion are emphasized by extending to the latter algorithms initially established for the former.     

Catadioptric Stereo Using Planar Mirrors

By using mirror reflections of a scene, stereo images can be captured with a single camera (catadioptric stereo). In addition to simplifying data acquisition single camera stereo provides both geometric and radiometric advantages over traditional two camera stereo. In this paper, we discuss the geometry and calibration of catadioptric stereo with two planar mirrors. In particular, we will show that the relative orientation of a catadioptric stereo rig is restricted to the class of planar motions thus reducing the number of external calibration parameters from 6 to 5. Next we derive the epipolar geometry for catadioptric stereo and show that it has 6 degrees of freedom rather than 7 for traditional stereo. Furthermore, we show how focal length can be recovered from a single catadioptric image solely from a set of stereo correspondences. To test the accuracy of the calibration we present a comparison to Tsai camera calibration and we measure the quality of Euclidean reconstruction. In addition, we will describe a real-time system which demonstrates the viability of stereo with mirrors as an alternative to traditional two camera stereo.     

Paracatadioptric camera calibration

Catadioptric sensors refer to the combination of lens-based devices and reflective surfaces. These systems are useful because they may have a field of view which is greater than hemispherical, providing the ability to simultaneously view in any direction. Configurations which have a unique effective viewpoint are of primary interest, among these is the case where the reflective surface is a parabolic mirror and the camera is such that it induces an orthographic projection and which we call paracatadioptric. We present an algorithm for the calibration of such a device using only the images of lines in space. In fact, we show that we may obtain all of the intrinsic parameters from the images of only three lines and that this is possible without any metric information. We propose a closed-form solution for focal length, image center, and aspect ratio for skewless cameras and a polynomial root solution in the presence of skew. We also give a method for determining the orientation of a plane containing two sets of parallel lines from one uncalibrated view. Such an orientation recovery enables a rectification which is impossible to achieve in the case of a single uncalibrated view taken by a conventional camera. We study the performance of the algorithm in simulated setups and compare results on real images with an approach based on the image of the mirror's bounding circle     

A new linear algorithm for calibrating central catadioptric cameras

In this paper, a novel linear calibration algorithm based on lines is presented for central catadioptric cameras. We firstly derive the relationship between the projection on the viewing sphere of a space point and its catadioptric image. And then by the relationship we establish a group of linear constraints on the catadioptric parameters from the catadioptric projections of spatial lines. By using these linear constraints, any central catadioptric camera can be fully calibrated from a single view of three or more lines without prior knowledge on the camera. Extensive experiments show this algorithm can improve the calibration's robustness.     

Geometric interpretations of the relation between the image of the absolute conic and sphere images

A spherical object has been introduced into camera calibration for several years through utilizing the properties of an image conic, which is the projection of the occluding contour of a sphere in the perspective image. However, in literature, only an algebraic interpretation was presented for the relation between the image of the absolute conic and sphere images. In this paper, we propose two geometric interpretations of this relation and two novel camera calibration methods using sphere images are derived from these geometric interpretations     

由共面圆确定摄像机参数的线性方法

考虑到基于二次曲线这种几何基元的摄像机标定方法比基于点或直线的方法具有更好的鲁棒性,给出了一种新的基于共面圆的摄像机标定方法。该方法的主要特点是模板形式简单、易于制作,仅需任意分布的三个或三个以上共面圆,且不需要进行圆环定位;不需要模板与图像之间的匹配;也无需求解任何非线性方程组。从几何角度对算法进行形象描述,并从代数的角度给出了严格论证。模拟和真实图像实验表明,该算法精确度高,鲁棒性强,表现出了十分良好的实用性。     

On improved calibration method for the catadioptric omnidirectional vision with a single viewpoint

The single viewpoint constraint is a principal optical characteristic for most catadioptric omnidirectional vision. Single viewpoint catadioptric omnidirectional vision is very useful because it allows the generation of geometrically correct perspective images from one omnidirectional image. Therefore precise calibration for single viewpoint constraint is needed during system assembling. However, in most image detection based calibration methods, the nonlinear optical distortion brought by lens is often neglected. Hence the calibration precision is poor. In this paper, a new calibration method of single viewpoint constraint for the catadioptric omni-directional vision is proposed. Firstly, an image correction algorithm is obtained by training a neural network. Then, according to characteristics of the space circular perspective projection, the corrected image of the mirror boundary is used to estimate its position and attitude relative to the camera to guide the calibration. Since the estimate is conducted based on actual imaging model rather than the simplified model, the estimate error is largely reduced, and the calibration accuracy is significantly improved. Experiments are conducted on simulated images and real images to show the accuracy and the effectiveness of the proposed method.