Capturing images with digital still cameras

We face multiple technical tasks when designing and developing digital cameras for consumer use. As would be expected, image quality is a top concern. Digital still cameras fall into two categories. One uses a black box and a camera lens with a CCD (charge-coupled device) back unit that can acquire more than 3 million pixels. Designed for professional and business use, this camera normally costs more than $5,000. The second type of digital camera is designed for consumers and employs a CCD capable of acquiring 350,000 to over 1 million pixels (Mpixels). Because it is easy to handle and carries a reasonable price for the picture quality, its market has grown dramatically in last three years. Here, I discuss the image-capturing system for a consumer-use digital still camera after briefly explaining its history     

Geometric properties of central catadioptric line images and their application in calibration

In central catadioptric systems lines in a scene are projected to conic curves in the image. This work studies the geometry of the central catadioptric projection of lines and its use in calibration. It is shown that the conic curves where the lines are mapped possess several projective invariant properties. From these properties, it follows that any central catadioptric system can be fully calibrated from an image of three or more lines. The image of the absolute conic, the relative pose between the camera and the mirror, and the shape of the reflective surface can be recovered using a geometric construction based on the conic loci where the lines are projected. This result is valid for any central catadioptric system and generalizes previous results for paracatadioptric sensors. Moreover, it is proven that systems with a hyperbolic/elliptical mirror can be calibrated from the image of two lines. If both the shape and the pose of the mirror are known, then two line images are enough to determine the image of the absolute conic encoding the camera's intrinsic parameters. The sensitivity to errors is evaluated and the approach is used to calibrate a real camera.     

Particle filtering with multiple and heterogeneous cameras

This work proposes a novel particle filter for tracking multiple people using multiple and heterogeneous cameras, namely monocular and stereo cameras. Our approach is to define confidence models and observation models for each type of camera. Particles are evaluated independently in each camera, and then the data are fused in accordance with the confidence. Confidence models take into account several sources of information. On the one hand, they consider occlusion information from an occlusion map calculated using a depth-ordered particle evaluation. On the other hand, the relative precision of sensors is considered so that the contribution of a sensor in the final data fusion step is proportional to its precision. We have defined confidence and observation models for monocular and stereo cameras and have designed tests to validate our proposal. The experiments show that our method is able to operate with each type individually and in combination. Two other remarkable properties of our method are that it is highly parallelizable and that it does not impose restrictions on the cameras’ positions or orientations.     

Dynamic calibration of pan-tilt-zoom cameras for traffic monitoring.

Pan-tilt-zoom (PTZ) cameras have been widely used in recent years for monitoring and surveillance applications. These cameras provide flexible view selection as well as a wider observation range. This makes them suitable for vision-based traffic monitoring and enforcement systems. To employ PTZ cameras for image measurement applications, one first needs to calibrate the camera to obtain meaningful results. For instance, the accuracy of estimating vehicle speed depends on the accuracy of camera calibration and that of vehicle tracking results. This paper presents a novel calibration method for a PTZ camera overlooking a traffic scene. The proposed approach requires no manual operation to select the positions of special features. It automatically uses a set of parallel lane markings and the lane width to compute the camera parameters, namely, focal length, tilt angle, and pan angle. Image processing procedures have been developed for automatically finding parallel lane markings. Interesting experimental results are presented to validate the robustness and accuracy of the proposed method.     

Frame-level temporal calibration of video sequences from unsynchronized cameras

This paper describes a method for temporally calibrating video sequences from unsynchronized cameras by image processing operations, and presents two search algorithms to match and align trajectories across different camera views. Existing multi-camera systems assume that input video sequences are synchronized either by genlock or by time stamp information and a centralized server. Yet, hardware-based synchronization increases installation cost. Hence, using image information is necessary to align frames from the cameras whose clocks are not synchronized. The system built for temporal calibration is composed of three modules: object tracking module, calibration data extraction module, and the search module. A robust and efficient search algorithm is introduced that recovers the frame offset by matching the trajectories in different views, and finding the most reliable match. Thanks to information obtained from multiple trajectories, this algorithm is robust to possible errors in background subtraction and location extraction. Moreover, the algorithm can handle very large frame offsets. A RANdom SAmple Consensus (RANSAC) based version of this search algorithm is also introduced. Results obtained with different video sequences are presented, which show the robustness of the algorithms in recovering various range of frame offsets for video sequences with varying levels of object activity.     

交通事故现场相机标定技术研究

在分析了交通事故现场处理特点的基础上,提出了基于棋盘格模板的照相机标定方法,即从不同角度拍摄多幅含有棋盘格模板的现场图像,在Tsai两步法的基础上先对相机进行线性标定,然后基于极大似然估计进行非线性优化,最后进行图像校正,获得照相机模型参数。实验结果表明该方法标定精度高,抗噪性能稳定,标定物简单,更加灵活易用。     

Functional calibration for pan-tilt-zoom cameras in hybrid sensor networks

Wide-area context awareness is a crucial enabling technology for next generation smart buildings and surveillance systems. It is not practical to gather this context awareness by covering the entire building with cameras. However, significant gaps in coverage caused by installing cameras in a sparse way can make it very difficult to infer the missing information. As a solution we advocate a class of hybrid perceptual systems that build a comprehensive model of activity in a large space, such as a building, by merging contextual information from a dense network of ultra-lightweight sensor nodes and video from a sparse network of cameras. In this paper we explore the task of automatically recovering the relative geometry between a pan-tilt-zoom camera and a network of one-bit motion detectors. We present results both for the recovery of geometry alone and also for the recovery of geometry jointly with simple activity models. Because we do not believe a metric calibration is necessary, or even entirely useful, for this task, we formulate and pursue the novel goal we term functional calibration. Functional calibration is a blending of geometry estimation and simple behavioral model discovery. Accordingly, results are evaluated by measuring the ability of the system to automatically foveate targets in a large, non-convex space, rather than by measuring, for example, pixel reconstruction error.     

A binarization method with learning-built rules for document images produced by cameras

In this paper, we propose a novel binarization method for document images produced by cameras. Such images often have varying degrees of brightness and require more careful treatment than merely applying a statistical method to obtain a threshold value. To resolve the problem, the proposed method divides an image into several regions and decides how to binarize each region. The decision rules are derived from a learning process that takes training images as input. Tests on images produced under normal and inadequate illumination conditions show that our method yields better visual quality and better OCR performance than three global binarization methods and four locally adaptive binarization methods.     

Radiance-based color calibration for image-based modeling with multiple cameras

Photo-consistency estimation is an important part for many image-based modeling techniques.This paper presents a novel radiance-based color calibration method to reduce the uncertainty of photo-consistency estimation across multiple cameras.The idea behind our method is to convert colors into a uniform radiometric color space in which multiple image data are corrected.Experimental results demonstrate that our method can achieve comparable color calibration effect without adjusting camera parameters and is more robust than other existing method.Additionally,we obtain an auto-determined threshold for photo-consistency check,which will lead to a better performance than existing photo-consistency based reconstruction algorithms.     

A variational approach to problems in calibration of multiple cameras

This paper addresses the problem of calibrating camera parameters using variational methods. One problem addressed is the severe lens distortion in low-cost cameras. For many computer vision algorithms aiming at reconstructing reliable representations of 3D scenes, the camera distortion effects will lead to inaccurate 3D reconstructions and geometrical measurements if not accounted for. A second problem is the color calibration problem caused by variations in camera responses that result in different color measurements and affects the algorithms that depend on these measurements. We also address the extrinsic camera calibration that estimates relative poses and orientations of multiple cameras in the system and the intrinsic camera calibration that estimates focal lengths and the skew parameters of the cameras. To address these calibration problems, we present multiview stereo techniques based on variational methods that utilize partial and ordinary differential equations. Our approach can also be considered as a coordinated refinement of camera calibration parameters. To reduce computational complexity of such algorithms, we utilize prior knowledge on the calibration object, making a piecewise smooth surface assumption, and evolve the pose, orientation, and scale parameters of such a 3D model object without requiring a 2D feature extraction from camera views. We derive the evolution equations for the distortion coefficients, the color calibration parameters, the extrinsic and intrinsic parameters of the cameras, and present experimental results.     

Optimal tiling of heterogeneous images

A novel information theoretical approach has been developed, implemented and tested to approximate large, heterogeneous images with maps of varying spatial resolution and predefinedcomplexity. The goal of the approximation is the derivation of units in the spatial domain rather than determining classes in the attribute (spectral) domain. The proposed procedure is a regular, hierarchical tiling. The value of each tile is the local mean and the parti tion leads to a map represented by a region quadtree with minimum Kullback-divergence from the original image. Kullback-divergence is a non-parametric measure of dissimilarity of two spatial distributions and is applied here because of several advantageous properties. This tiling procedure can be also viewed as data compression, and it optimizes information loss under constraint on the spatial arrangement and number of tiles. The methodology is illustrated by the sampling design of an environmental soil mapping project of the salt-affected rangeland in Hortobagy, north-east Hungary.     

Camera calibration from images of spheres

This paper introduces a novel approach for solving the problem of camera calibration from spheres. By exploiting the relationship between the dual images of spheres and the dual image of the absolute conic (IAC), it is shown that the common pole and polar with regard to the conic images of two spheres are also the pole and polar with regard to the IAC. This provides two constraints for estimating the IAC and, hence, allows a camera to be calibrated from an image of at least three spheres. Experimental results show the feasibility of the proposed approach     

Efficient generic calibration method for general cameras with single centre of projection

Generic camera calibration is a non-parametric calibration technique that is applicable to any type of vision sensor. However, the standard generic calibration method was developed such that both central and non-central cameras can be calibrated within the same framework. Consequently, existing parametric calibration techniques cannot be applied for the common case of cameras with a single centre of projection (e.g. pinhole, fisheye, hyperboloidal catadioptric). This paper proposes improvements to the standard generic calibration method for central cameras that reduce its complexity, and improve its accuracy and robustness. Improvements are achieved by taking advantage of the geometric constraints resulting from a single centre of projection in order to enable the application of established pinhole calibration techniques. Input data for the algorithm is acquired using active grids, the performance of which is characterised. A novel linear estimation stage is proposed that enables a well established pinhole calibration technique to be used to estimate the camera centre and initial grid poses. The proposed solution is shown to be more accurate than the linear estimation stage of the standard method. A linear alternative to the existing polynomial method for estimating the pose of additional grids used in the calibration is demonstrated and evaluated. Distortion correction experiments are conducted with real data for both an omnidirectional camera and a fisheye camera using the standard and proposed methods. Motion reconstruction experiments are also undertaken for the omnidirectional camera. Results show the accuracy and robustness of the proposed method to be improved over those of the standard method.     

一种针孔相机与三维激光雷达外参标定方法

针对针孔相机和三维激光雷达的外参标定问题,提出了一种新的方法:利用相机坐标系和激光坐标系下棋盘面的对应性,将外参标定转换为三维空间中旋转、缩放矩阵的求解问题.每帧数据可以提供4个约束,最少只需3帧数据可求解外参矩阵.方法原理简单易懂,直观明了地解释了失效退化原因.仿真实验与真实数据实验表明:提出的方法可以得到高精度的外参矩阵,且在采样帧数较少的情况下依旧可以获得很好的标定效果.     

Color reconstruction in digital cameras: optimization for document images

Digital cameras normally sample one color at each pixel. Missing colors are obtained by spatial interpolation, decreasing resolution relative to images acquired with a greyscale sensor. The consequence for document imaging is higher text recognition error rates. This paper introduces the horizontal-vertical regression (HVR) method for document-optimized color reconstruction. HVR exploits a local two-color approximation, making spatial interpolation unnecessary. Comparison with the best alternative reconstruction methods indicates large reductions in error rates for text resulting from HVR, as well as improvements in intermediate color and binary images.Received: 11 June 2003, Accepted: 6 March 2004, Published online: 2 February 2005C. Dance: Correspondence to     

基于畸变校正的双目立体摄像机线性标定

利用透视变换原理建立双目立体摄像机数学模型,全面考虑了镜头的径向畸变和切向畸 变,提出一种线性求解摄像机参数的标定方法,改变了以往的摄像机标定依赖于非线性优化的缺点,避免了非线性优化的不稳定性.该标定方法在单摄像机模型的基础上,加入对双摄像机相对位置的确定,通过成像过程中坐标系之间的转换,较好地实现了双目立体摄像测量系...     

基于边缘关联点云的激光雷达与相机外参标定方法

激光雷达的点云和相机的图像经常被融合应用在多个领域。准确的外参标定是融合两者信息的前提。点云特征提取是外参标定的关键步骤。但是点云的低分辨率和低质量会影响标定结果的精度。针对这些问题,提出一种基于边缘关联点云的激光雷达与相机外参标定方法。首先,利用双回波提取标定板边缘关联点云;然后,通过优化方法从边缘关联点云中提取出与实际标定板尺寸大小兼容的标定板角点;最后,将点云中角点和图像中角点匹配。用多点透视方法求解激光雷达与相机之间的外参。实验结果表明,该方法的重投影误差为1.602px,低于同类对比方法,验证了该方法的有效性与准确性。     

Optimizing PTZ camera calibration from two images

In this paper, we address the problem of calibrating an active pan–tilt–zoom (PTZ) camera. In this regard, we make three main contributions: first, for the general camera rotation, we provide a novel solution that yields four independent constraints from only two images, by directly decomposing the infinite homography using a series of Givens rotations. Second, for a camera varying its focal length, we present a solution for the degenerate cases of pure pan and pure tilt that occur very frequently in practical applications of PTZ cameras. Third, we derive a new optimized error function for pure rotation or pan–tilt rotation, which plays a similar role as the epipolar constraint in a freely moving camera, in terms of characterizing the reprojection error of point correspondences. Our solutions and analysis are thoroughly validated and tested on both synthetic and real data, whereby the new geometric error function is shown to outperform existing methods in terms of accuracy and noise resilience.