Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment

The advent of high-resolution digital cameras and sophisticated multi-view stereo algorithms offers the promise of unprecedented geometric fidelity in image-based modeling tasks, but it also puts unprecedented demands on camera calibration to fulfill these promises. This paper presents a novel approach to camera calibration where top-down information from rough camera parameter estimates and the output of a multi-view-stereo system on scaled-down input images is used to effectively guide the search for additional image correspondences and significantly improve camera calibration parameters using a standard bundle adjustment algorithm (Lourakis and Argyros 2008). The proposed method has been tested on six real datasets including objects without salient features for which image correspondences cannot be found in a purely bottom-up fashion, and objects with high curvature and thin structures that are lost in visual hull construction even with small errors in camera parameters. Three different methods have been used to qualitatively assess the improvements of the camera parameters. The implementation of the proposed algorithm is publicly available at Furukawa and Ponce (2008b).     

3D Scene Reconstruction from Multiple Spherical Stereo Pairs

We propose a 3D environment modelling method using multiple pairs of high-resolution spherical images. Spherical images of a scene are captured using a rotating line scan camera. Reconstruction is based on stereo image pairs with a vertical displacement between camera views. A 3D mesh model for each pair of spherical images is reconstructed by stereo matching. For accurate surface reconstruction, we propose a PDE-based disparity estimation method which produces continuous depth fields with sharp depth discontinuities even in occluded and highly textured regions. A full environment model is constructed by fusion of partial reconstruction from spherical stereo pairs at multiple widely spaced locations. To avoid camera calibration steps for all camera locations, we calculate 3D rigid transforms between capture points using feature matching and register all meshes into a unified coordinate system. Finally a complete 3D model of the environment is generated by selecting the most reliable observations among overlapped surface measurements considering surface visibility, orientation and distance from the camera. We analyse the characteristics and behaviour of errors for spherical stereo imaging. Performance of the proposed algorithm is evaluated against ground-truth from the Middlebury stereo test bed and LIDAR scans. Results are also compared with conventional structure-from-motion algorithms. The final composite model is rendered from a wide range of viewpoints with high quality textures.     

一种基于校正误差的立体相机标定算法

立体相机的标定是一个精确求解各个相机内参数以及相机之间关系参数的过程.它是三维重建的基础,其标定精度的好坏直接影响立体重建的结果.为此提出了一种使用校正误差作为代价函数的立体相机标定算法.该算法首先使用传统的基于重投影误差的方法对单个相机的内参数进行标定,然后利用校正误差完成对相机之间关系参数的标定求解.由于校正误差的计算只与相机内参数以及关系参数有关,可以避免在标定过程中使用难以精确标定的相机外参数.实验结果表明本算法能够有效的提高立体相机标定的精度.     

一种基于单相机三维重建的简单方法

提出了一种基于单相机三维重建的简单方法.在待重建场景中放置一块平面标定模板,用内参数已知的摄像机拍摄不同角度的二幅图像,利用场景中的标定模板精确地求解出拍摄时摄像机的外参数,并以此为基础进行立体标定、配准和匹配,得到视差图进而实现欧式几何意义下的三维重建.实验结果表明,提出的算法实现简单、配置灵活,具有较强的实用性.     

改进格雷码结构光双目视觉工件3D测量

机器人喷涂、焊接、研磨等现代制造过程,通常需要获取工件3D几何形状信息以实现路径自动规划。针对此需求,提出了一种改进灰度编码的结构光双目视觉成像与三维测量系统。在离线标定获得相机参数的基础上,采用黑白反向重投影方法解决传统格雷码结构光成像中边界模糊问题;同时,提出快速区域块搜索算法,用于立体匹配和视差图生成;最后,采用Lanczos插值算法填补遮挡造成的视差图空白,从而获得完整的深度信息,实现工件高精度3D重建。实际工件成像实验结果表明：该方法能够实现工件快速、高精度三维重建,尤其对无纹理和弱纹理目标能获得较好的结果,能够满足工业环境测量需求。     

Camera calibration for a mobile robot prototype

Stereovision is an effective technique to use a CCD video camera to determine the 3D position of a target object from two or more simultaneous views of the scene. Camera calibration is a central issue in finding the position of objects in a stereovision system. This is usually carried out by calibrating each camera independently, and then applying a geometric transformation of the external parameters to find the geometry of the stereo setting. After calibration, the distance of various target objects in the scene can be calculated with CCD video cameras, and recovering the 3D structure from 2D images becomes simpler. However, the process of camera calibration is complicated. Based on the ideal pinhole model of a camera, we describe formulas to calculate intrinsic parameters that specify the correct camera characteristics, and extrinsic parameters that describe the spatial relationship between the camera and the world coordinate system. A simple camera calibration method for our CCD video cameras and corresponding experiment results are also given. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

Non-goal scene analysis for soccer video

The broadcast soccer video is usually recorded by one main camera, which is constantly gazing somewhere of playfield where a highlight event is happening. So the camera parameters and their variety have close relationship with semantic information of soccer video, and much interest has been caught in camera calibration for soccer video. The previous calibration methods either deal with goal scene, or have strict calibration conditions and high complexity. So, it does not properly handle the non-goal scene such as midfield or center-forward scene. In this paper, based on a new soccer field model, a field symbol extraction algorithm is proposed to extract the calibration information. Then a two-stage calibration approach is developed which can calibrate camera not only for goal scene but also for non-goal scene. The preliminary experimental results demonstrate its robustness and accuracy.     

高速公路云台相机的自动标定

目的 云台相机因监控视野广、灵活度高,在高速公路监控系统中发挥出重要的作用,但因云台相机焦距与角度不定时地随监控需求变化,对利用云台相机的图像信息获取真实世界准确的物理信息造成一定困难,因此进行云台相机非现场自动标定方法的研究对高速公路监控系统的应用具有重要价值。方法 本文提出了一种基于消失点约束与车道线模型约束的云台相机自动标定方法,以建立高速公路监控系统的图像信息与真实世界物理信息之间准确描述关系。首先,利用车辆目标运动轨迹的级联霍夫变换投票实现纵向消失点的准确估计,其次以车道线模型物理度量为约束,并采用枚举策略获取横向消失点的准确估计,最终在已知相机高度的条件下实现高速公路云台相机标定参数的准确计算。结果 将本文方法在不同的场景下进行实验,得到在不同的距离下的平均误差分别为4.63%、4.74%、4.81%、4.65%,均小于5%。结论 对多组高速公路监控场景的测试实验结果表明,本文提出的云台相机自动标定方法对高速公路监控场景的物理测量误差能够满足应用需求,与参考方法相比较而言具有较大的优势和一定的应用价值,得到的相机内外参数可用于计算车辆速度与空间位置等。     

Catadioptric Stereo Using Planar Mirrors

By using mirror reflections of a scene, stereo images can be captured with a single camera (catadioptric stereo). In addition to simplifying data acquisition single camera stereo provides both geometric and radiometric advantages over traditional two camera stereo. In this paper, we discuss the geometry and calibration of catadioptric stereo with two planar mirrors. In particular, we will show that the relative orientation of a catadioptric stereo rig is restricted to the class of planar motions thus reducing the number of external calibration parameters from 6 to 5. Next we derive the epipolar geometry for catadioptric stereo and show that it has 6 degrees of freedom rather than 7 for traditional stereo. Furthermore, we show how focal length can be recovered from a single catadioptric image solely from a set of stereo correspondences. To test the accuracy of the calibration we present a comparison to Tsai camera calibration and we measure the quality of Euclidean reconstruction. In addition, we will describe a real-time system which demonstrates the viability of stereo with mirrors as an alternative to traditional two camera stereo.     

双目视觉测量系统摄像机外部参数标定研究

针对双目立体视觉测量系统中摄像机标定问题,讨论了基于标准长度的外部参数标定方法,选定了摄像机透视投影模型,采用双摄像机同时对放置于视场内的十字靶标拍摄多幅图像,得出了基于LabVIEW开发的摄像机标定方法.该方法利用了LabVIEW的开发环境,使用了数学工具包,将遗传算法与LM算法相结合,优化迭代获得摄像机外部参数,运算速度和精度大大提高.开发的模块可用于基于LabVIEW开发的工程软件进行高精度尺寸现场测量.在双目立体视觉测量系统标定结果基础上对标准靶进行测量,测量结果标准差达到0.1.     

STEREO CAMERA CALIBRATION USING PARTICLE SWARM OPTIMIZATION

Camera calibration is an essential issue in many computer vision tasks in which quantitative information of a scene is to be derived from its images. It is concerned with the determination of a set of parameters from the given images. In literature, it has been modeled as a nonlinear global optimization problem and has been solved using various optimization techniques. In this article, a recently developed variant of a very popular global optimization technique—the particle swarm optimization (PSO) algorithm—has been used for solving this problem for a stereo camera system modeled by pin-hole camera model. Extensive experiments have been performed on synthetic data to test the applicability of the technique to this problem. The simulation results, which have been compared with those obtained by a real coded genetic algorithm (RCGA) in literature, show that the proposed PSO performs a bit better than RCGA in terms of computational effort.     

基于单幅图片的相机完全标定

现有相机标定方法的标定过程比较繁琐,不利于标定相机的广泛使用。为此,从摄像机镜头畸变矫正着手,结合标定板信息及消失点约束,提出一种基于单张图片的相机标定方法。利用非线性迭代得到相机镜头的畸变系数,通过线性求解得出相机的内参,直接计算得到相机的外参,从而实现仅需拍摄单张标定板图片的相机完全标定。实验结果表明,该方法在标定板与视平面夹角小于45°的情况下均能成功标定,并且重投影误差小于0.3像素。     

一种多尺度几何分析的摄像机对弱定标算法

自主车上的立体视觉系统一般由两台固定在平台上的定焦摄像机组成,因此摄像机内参数经一次标定后不再变化,只需要考虑外参数的标定.本文针对自主车视觉系统的特殊应用情况,提出一种基于多尺度几何分析思想的摄像机对弱定标算法,该算法采用Contourlet变换对左右图像中的角点进行检测,利用Hartley规范化8点法估计摄像机对的基础矩阵.依托现有的摄像机内部参数标定工具箱,在摄像机对弱定标的基础上还可以快速地获得摄像机对之间的外参数矩阵.实验结果表明该方法具有较好的精度.     

一种新的基于散焦图像的摄像机标定方法

提出一种新的用于散焦求深度的摄像机内参数标定算法。该算法依靠改变摄像机镜头光圈指数获取同一场景的任意两幅散焦程度不同的图像,提取两幅图像间模糊程度差异信息,结合分析透镜成像几何标定出摄像机的相应内参数。此算法解除了2006年由Park所提出的标定方法中必须有一幅聚焦图像的限制,并且无须对图像进行复杂的放大率标准化处理。模拟实验与真实实验均验证了算法的有效性和精确性。     

Multi-object detection and tracking by stereo vision

This paper presents a new stereo vision-based model for multi-object detection and tracking in surveillance systems. Unlike most existing monocular camera-based systems, a stereo vision system is constructed in our model to overcome the problems of illumination variation, shadow interference, and object occlusion. In each frame, a sparse set of feature points are identified in the camera coordinate system, and then projected to the 2D ground plane. A kernel-based clustering algorithm is proposed to group the projected points according to their height values and locations on the plane. By producing clusters, the number, position, and orientation of objects in the surveillance scene can be determined for online multi-object detection and tracking. Experiments on both indoor and outdoor applications with complex scenes show the advantages of the proposed system.     

Auto‐calibration of a projector–camera stereo system for projection mapping

Standard camera and projector calibration techniques use a checkerboard that is manually shown at different poses to determine the calibration parameters. Furthermore, when image geometric correction must be performed on a three‐dimensional (3D) surface, such as projection mapping, the surface geometry must be determined. Camera calibration and 3D surface estimation can be costly, error prone, and time‐consuming when performed manually. To address this issue, we use an auto‐calibration technique that projects a series of Gray code structured light patterns. These patterns are captured by the camera to build a dense pixel correspondence between the projector and camera, which are used to calibrate the stereo system using an objective function, which embeds the calibration parameters together with the undistorted points. Minimization is carried out by a greedy algorithm that minimizes the cost at each iteration with respect to both calibration parameters and noisy image points. We test the auto‐calibration on different scenes and show that the results closely match a manual calibration of the system. We show that this technique can be used to build a 3D model of the scene, which in turn with the dense pixel correspondence can be used for geometric screen correction on any arbitrary surface.     

Joint Estimation of Shape and Reflectance using Multiple Images with Known Illumination Conditions

We propose a generative model based method for recovering both the shape and the reflectance of the surface(s) of a scene from multiple images, assuming that illumination conditions and cameras calibration are known in advance. Based on a variational framework and via gradient descents, the algorithm minimizes simultaneously and consistently a global cost functional with respect to both shape and reflectance. The motivations for our approach are threefold. (1) Contrary to previous works which mainly consider specific individual scenarios, our method applies indiscriminately to a number of classical scenarios; in particular it works for classical stereovision, multiview photometric stereo and multiview shape from shading. It works with changing as well as static illumination. (2) Our approach naturally combines stereo, silhouette and shading cues in a single framework. (3) Moreover, unlike most previous methods dealing with only Lambertian surfaces, the proposed method considers general dichromatic surfaces. We verify the method using various synthetic and real data sets.     

基于EXIF信息的可变内参数序列图像的三维重构*

基于图像中数码相机嵌入的元数据信息,提出了一种可变内参数的序列图像重构算法,首先从序列中选择两幅图像建立初始结构,然后依次将其他图像加入当前重构结果,进一步通过集束调整来最小化序列中所有图像重投影误差,得到精确的三维重构结果,避免了复杂繁琐的自标定过程。实验结果验证了算法的有效性。     

Adaptive skin color modeling using the skin locus for selecting training pixels

Techniques for color-based tracking of faces or hands often assume a static skin model yet skin color, as measured by a camera, can change when lighting changes. Therefore, for robust skin pixel detection, an adaptive skin color model must be employed. We demonstrate a chromaticity-based constraint to select training pixels in a scene for updating a dynamic skin color model under changing illumination conditions. The method makes use of the ‘skin locus’ of a camera, that is, the area in chromaticity space where skin chromaticity under various lighting and camera calibration conditions is observed. Skin color models derived from the technique are compared with that derived by a common spatial constraint and is shown to be more consistent with manually extracted ground truth skin model per frame even as localization errors increase. The technique is applied to color-based face tracking in indoor and outdoor videos and is shown to succeed more often than other color model adaptation techniques.     

大视场双目立体视觉的摄像机标定

针对大视场视觉测量应用,在分析摄像机成像模型的基础上,设计制作了可自由转动的十字靶标,实现了大视场双目视觉摄像机的精确标定。将十字靶标在测量空间内多次均匀摆放,两摄像机同步拍摄多幅靶标图像。由本质矩阵得到摄像机参数的初始值,采用自检校光束法平差得到摄像机参数的最优解。该方法不要求特征点共面,仅需要知道特征点之间的物理距离,降低了靶标制作难度。采用TN3DOMS．S进行了实测,在1500mm×1500mm的测量范围内测试标准标杆,误差均方值为0．06mm。