Accurate recovery of three-dimensional shape from image focus

A new shape-from-focus method is described which is based on a new concept, named focused image surface (FIS). FIS of an object is defined as the surface formed by the set of points at which the object points are focused by a camera lens. According to paraxial-geometric optics, there is a one-to-one correspondence between the shape of an object and the shape of its FIS. Therefore, the problem of shape recovery can be posed as the problem of determining the shape of the FIS. From the shape of FIS the shape of the object is easily obtained. In this paper the shape of the FIS is determined by searching for a shape which maximizes a focus measure. In contrast to previous literature where the focus measure is computed over the planar image detector of the camera, here the focus measure is computed over the FIS. This results in more accurate shape recovery than the traditional methods. Also, using FIS, a more accurate focused image can be reconstructed from a sequence of images than is possible with traditional methods. The new method has been implemented on an actual camera system, and the results of shape recovery and focused image reconstruction are presented     

Real-time camera tracking using sports pitch markings

When broadcasting sports events such as football, it is useful to be able to place virtual annotations on the pitch, to indicate things such as distances between players and the goal, or whether a player is offside. This requires the camera position, orientation, and focal length to be estimated in real time, so that the graphics can be rendered to match the camera view. Whilst this can be achieved by using sensors on the camera mount and lens, they can be impractical or expensive to install, and often the broadcaster only has access to the video feed itself. This paper presents a method for computing the position, orientation and focal length of a camera in real time, using image analysis. The method uses markings on the pitch, such as arcs and lines, to compute the camera pose. A novel feature of the method is the use of multiple images to improve the accuracy of the camera position estimate. A means of automatically initialising the tracking process is also presented, which makes use of a modified form of Hough transform. The paper shows how a carefully chosen set of algorithms can provide fast, robust and accurate tracking for this real-world application.     

Selecting the optimal focus measure for autofocusing anddepth-from-focus

A method is described for selecting the optimal focus measure with respect to gray-level noise from a given set of focus measures in passive autofocusing and depth-from-focus applications. The method is based on two new metrics that have been defined for estimating the noise-sensitivity of different focus measures. The first metric-the autofocusing uncertainty measure (AUM)-is useful in understanding the relation between gray-level noise and the resulting error in lens position for autofocusing. The second metric-autofocusing root-mean-square error (ARMS error)-is an improved metric closely related to AUM. AUM and ARMS error metrics are based on a theoretical noise sensitivity analysis of focus measures, and they are related by a monotonic expression. The theoretical results are validated by actual and simulation experiments. For a given camera, the optimally accurate focus measure may change from one object to the other depending on their focused images. Therefore, selecting the optimal focus measure from a given set involves computing all focus measures in the set     

基于空间透视不变量的摄象机标定方法

本文给出了一种以空间不变量的数据来计算摄象机外部参数的方法．空间透视不变量是指在几何变换中如投影或改变观察点时保持不变的形状描述．由于它可以得到一个相对于外界来讲独立的物体景物的特征描述，故可以很广泛的应用到计算机视觉等方面．摄象机标定是确定摄象机摄取的２Ｄ图象信息及其３Ｄ实际景物的信息之间的变换关系，它包括内部参数和外部参数两个部分．内部参数表征的是摄象机的内部特征和光学特征参数，包括图象中心（Ｃｘ，Ｃｙ）坐标、图象尺度因子Ｓｘ、有效的焦距长度ｆ和透镜的畸变失真系数Ｋ；外部参数表示的是摄象机的位置和方向在世界坐标中的坐标参数，它包括平移矩阵Ｔ和旋转矩阵Ｒ３×３，一般情况下可以写成一个扩展矩阵［ＲＴ］３×４．本文基于空间透视不变量的计算数据，给出了一种标定摄象机外部参数的方法，实验结果表明该方法具有很强的鲁棒性．     

Virtual wiper — removal of adherent noises from images of dynamic scenes by using a pan–tilt camera

In this paper, we propose a new method that can remove noises from images of dynamic scenes that can cause viewing problems. One of the thorny problems in outdoor surveillance by a camera is that adherent noises such as waterdrops or mud blobs on the protecting glass surface lens disturb the view from the camera. Therefore, we propose a method for removing adherent noises from images of dynamic scenes taken by changing the direction of a pan–tilt camera, which is often used for surveillance. Our method is based on the comparison of two images — a reference image and a second image taken by a different camera angle. The latter image is transformed by a projective transformation and subtracted from the reference image to extract the regions of adherent noises and moving objects. The regions of adherent noises in the reference image are identified by examining the shapes and distances of regions existing in the subtracted image. Finally, regions of adherent noises can be eliminated by merging two images. Experimental results show the effectiveness of our proposed method.     

A method of reactive zoom control from uncertainty in tracking

The tuning of a constant velocity Kalman filter, used for tracking by a camera fitted with a variable focal-length lens, is shown to be preserved under a scale change in process noise if accompanied by an inverse scaling in the focal length, provided the image measurement error is of fixed size in image coordinates. Based on this observation, a practical method of zoom control has been built by setting an upper limit on the probability that the innovation (and hence fixation error) exceeds the image half-width. The innovation covariance matrix used to determine the innovation limit is derived over two timescales, which enables a rapid zooming out response and slower zooming in. Experimental simulations are presented, before results are given from a video-rate implementation using a camera with two motorized orientation axes and fitted with a computer-controlled zoom lens. The delays in the feedback loops, comprising image capture delay, platform response lag and zoom lens response lag, are carefully calibrated by fitting to their frequency responses. It is found that the cumulative uncertainty in delay gives rise to an image error which is part constant and part proportional to focal length, resulting in a beneficial adaptation of the filter.     

Accurate Depth Dependent Lens Distortion Models: An Application to Planar View Scenarios

In order to calibrate cameras in an accurate manner, lens distortion models have to be included in the calibration procedure. Usually, the lens distortion models used in camera calibration depend on radial functions of image pixel coordinates. Such models are well-known, simple and can be estimated using just image information. However, these models do not take into account an important physical constraint of lens distortion phenomena, namely: the amount of lens distortion induced in an image point depends on the scene point depth with respect to the camera projection plane. In this paper we propose a new accurate depth dependent lens distortion model. To validate this approach, we apply the new lens distortion model to camera calibration in planar view scenarios (that is 3D scenarios where the objects of interest lie on a plane). We present promising experimental results on planar pattern images and on sport event scenarios. Nevertheless, although we emphasize the feasibility of the method for planar view scenarios, the proposed model is valid in general and can be used in any scenario where the point depth can be estimated.     

Camera calibration from surfaces of revolution

This paper addresses the problem of calibrating a pinhole camera from images of a surface of revolution. Camera calibration is the process of determining the intrinsic or internal parameters (i.e., aspect ratio, focal length, and principal point) of a camera, and it is important for both motion estimation and metric reconstruction of 3D models. In this paper, a novel and simple calibration technique is introduced, which is based on exploiting the symmetry of images of surfaces of revolution. Traditional techniques for camera calibration involve taking images of some precisely machined calibration pattern (such as a calibration grid). The use of surfaces of revolution, which are commonly found in daily life (e.g., bowls and vases), makes the process easier as a result of the reduced cost and increased accessibility of the calibration objects. In this paper, it is shown that two images of a surface of revolution will provide enough information for determining the aspect ratio, focal length, and principal point of a camera with fixed intrinsic parameters. The algorithms presented in this paper have been implemented and tested with both synthetic and real data. Experimental results show that the camera calibration method presented is both practical and accurate.     

Less restrictive camera odometry estimation from monocular camera

This paper addresses the problem of estimating a camera motion from a non-calibrated monocular camera. Compared to existing methods that rely on restrictive assumptions, we propose a method which can estimate camera motion with much less restrictions by adopting new example-based techniques compensating the lack of information. Specifically, we estimate the focal length of the camera by referring to visually similar training images with which focal lengths are associated. For one step camera estimation, we refer to stationary points (landmark points) whose depths are estimated based on RGB-D candidates. In addition to landmark points, moving objects can be also used as an information source to estimate the camera motion. Therefore, our method simultaneously estimates the camera motion for a video, and the 3D trajectories of objects in this video by using Reversible Jump Markov Chain Monte Carlo (RJ-MCMC) particle filtering. Our method is evaluated on challenging datasets demonstrating its effectiveness and efficiency.     

An on-line optical bench tester machine for evaluating lens quality

This study is dedicated to develop an on-line automatic optical bench tester (OBT) machine for evaluating the image quality of a camera lens that is used in a lens module of a cell phone. This tester is not only suitable for conventional solid lens, but also applicable to the developing cutting-edge tunable liquid crystal lens. The testing is accomplished via a specially-designed OBT machine, which is able to automatically move the test lens based on feedback images in the optical system in the OBT to the axial position that leads to best imaging quality and also successfully measure its focus length. In the designed OBT, a commercial inspection chart is employed, along with an automatic lens-feeding machine for a quick estimate on the best possible focusing quality, which is evaluated by the well-known modulation transfer function (MTF). For actuating the feeding machine, an algorithm, assisted by the feedback MTFs, is proposed to move the test lens to the particular position that renders the best quality. In this way, the focus length—effective focal length (EFL)—of the test lens can be obtained. The proposed algorithm in fact needs much less time of actuation than a traditional tester to obtain EFL of the test lens. The designed and constructed tester is capable of measuring varied optical performance indices for the next-generation tunable lens, like liquid crystal lens.     

基于散焦图像深度测量的一种新方法

提出了基于散焦图像深度测量的一种新方法．该方法采用一台带有远心镜头的CCD摄 像机,沿光轴方向移动摄像机拍摄两副图像,根据所拍摄目标图像的散焦半径与图像的大小 计算目标景物距摄像机的距离．采用远心光学镜头代替普通镜头可使图像的大小与散焦半径 之间的关系简单．该方法融合了图像的大小与散焦半径两种信息,使深度计算更加准确．由 于该方法只需要一台参数固定的CCD摄像机,可以免除图像间的配准和特征点的选取,有利 于实时系统的实现．实验结果表明该方法的有效性．     

内参数异构情况下摄像机平移位置的测定

在手眼系统中,摄像机平移是比较容易实现和经常需要的一种运动.本文首先给出了摄像机调整焦距 后,焦距标定的两个比较简捷的方法.然后,根据平移运动中摄像机获取的两幅不同图像,在焦距不同的情况下,给 出了检测摄像机坐标系平行移动的方向的方法.事实上,摄像机调整焦距也是一种平移运动,因此这时的摄像机的 平移运动是两种平移运动的综合的结果,同时给出了由平台平移引起的摄像机坐标系平移位置的求解方法.     

Characterization of Errors in Compositing Panoramic Images

A panoramic image has a 360° horizontal field of view, and it can provide the viewer the impression of being immersed in the scene. A panorama is created by first taking a sequence of images while rotating the camera about a vertical axis. These images are then projected onto a cylindrical surface before being seamlessly composited. The cross-sectional circumference of the cylindrical panorama is called thecompositing length. This work characterizes the error in compositing panoramic images due to errors in some of the intrinsic parameters. The intrinsic camera parameters that are considered are the camera focal length and the radial distortion coefficient. We show that the error in the compositing length is more sensitive to the error in the camera focal length. Especially important is the discovery that the relative error in compositing length is always smaller than the relative error in the focal length. This means that the error in focal length can be corrected by iteratively using the composited length to compute a new and more correct focal length. Thiscompositing approach to camera calibrationhas the advantages of not requiring both feature detection and separate prior calibration.     

基于圆柱面映射的快速图像拼接算法

针对现有基于圆柱面映射的全景图像拼接算法无法实现自动估计焦距的问题,为满足实时性要求,提出了一种基于预测的快速特征点匹配算法,在基于纯旋转运动的自动焦距估计算法基础上,提出了一种基于单应矩阵的焦距修正算法。该算法首先从待拼接图像中提取Harris角点,然后提取方向梯度直方图(Histogram of Oriented Gradient, HOG)描述子进行特征点匹配,基于前一帧图像获得的图像映射关系采用一种基于预测的快速特征点匹配算法,然后使用简化的基于纯旋转运动的焦距估计算法估计出焦距初值,并采用基于单应矩阵的焦距修正算法得到更精确的焦距值,最后将平面图像投影至圆柱平面,使用加权平均融合算法进行拼接,合成全景图像。采用多个测试序列图像对算法进行测试,特征点匹配速度较传统方法提高了10倍以上,自动焦距估计算法能够准确估计摄像机焦距,且耗时仅50毫秒左右。实验结果表明,提出的算法能够快速地合成高质量的全景图像,拼接后的图像畸变小,具有较高的实用价值。     

Computing camera parameters using vanishing-line information from a rectangular parallelepiped

A new approach to camera calibration using vanishing line information for three-dimensional computer vision is proposed. Calibrated parameters include the orientation, the position, the focal length, and the image plane center of a camera. A rectangular parallelepiped is employed as the calibration target to generate three principal vanishing points and then three vanishing lines from the projected image of the parallelepiped. Only a monocular image is required for solving these camera parameters. It is shown that the image plane center is the orthocenter of a triangle formed by the three vanishing lines. From the slopes of the vanishing lines the camera orientation parameters can be determined. The focal length can be computed by the area of the triangle. The camera position parameters can then be calibrated by using related geometric projective relationships. The derived results show the geometric meanings of these camera parameters. The calibration formulas are analytic and simple to compute. Experimental results show the feasibility of the proposed approach for a practical application—autonomous land vehicle guidance.This work was supported by National Science Council, Republic of China under Grant NSC-77-0404-E-009-31.     

Camera re-calibration after zooming based on sets of conics

We describe a method to compute the internal parameters (focal and principal point) of a camera with known position and orientation, based on the observation of two or more conics on a known plane. The conics can even be degenerate (e.g., pairs of lines). The proposed method can be used to re-estimate the internal parameters of a fully calibrated camera after zooming to a new, unknown, focal length. It also allows estimating the internal parameters when a second, fully calibrated camera observes the same conics. The parameters estimated through the proposed method are coherent with the output of more traditional procedures that require a higher number of calibration images. A deep analysis of the geometrical configurations that influence the proposed method is also reported.     

Recursive 3-D Visual Motion Estimation Using Subspace Constraints

A structure from motion algorithm is described which recovers structure and camera position, modulo a projective ambiguity. Camera calibration is not required, and camera parameters such as focal length can be altered freely during motion. The structure is updated sequentially over an image sequence, in contrast to schemes which employ a batch process. A specialisation of the algorithm to recover structure and camera position modulo an affine transformation is described, together with a method to periodically update the affine coordinate frame to prevent drift over time. We describe the constraint used to obtain this specialisation.Structure is recovered from image corners detected and matched automatically and reliably in real image sequences. Results are shown for reference objects and indoor environments, and accuracy of recovered structure is fully evaluated and compared for a number of reconstruction schemes. A specific application of the work is demonstrated—affine structure is used to compute free space maps enabling navigation through unstructured environments and avoidance of obstacles. The path planning involves only affine constructions.     

Cylindrical panoramic mosaicing from a pipeline video through MRF based optimization

Stratum structure detection is a fundamental problem in geological engineering. One of the most commonly employed detection technologies is to shoot videos of a borehole using a forward moving camera. Using this technology, the problem of stratum structure detection is transformed into the problem of constructing a panoramic image from a low quality video. In this paper, we propose a novel method for creating a panoramic image of a borehole from a video sequence without the need of camera calibration and tracking. To stitch together pixels of neighboring image frames, our camera model is designed with a focal length changing feature, along with a small rotational freedom in the two-dimensional image space. Our camera model assumes that target objects lie on a cylindrical wall and that the camera moves forward along the central axis of the cylindrical wall. Based on these two assumptions, our method robustly resolves these two degrees-of-freedoms in our camera model through KLT feature tracking. Since the quality of the result video is affected by possible illumination overflow, camera lens blurring, and low video resolution, we introduce a cost function for eliminating seams between stitching strips. Our cost function is designed based on Markov Random Field and optimized using a belief propagation algorithm. Using our method, we can automatically construct a panorama image with good resolution, smoothness, and continuousness both in the texture and illumination space. Experiment results show that our method could efficiently generate panoramas for long video sequences with satisfying visual quality.     

有源加密型微光学标签系统的设计

通过添加辅助光学元件,并采用OOK调制加密方式,给出了一种有源加密型微光学标签的光学系统设计方法。该方法相比一般微光学标签系统,接收端采用了更为普及的手机相机,并具有标签加密等特性;同时根据手机相机镜头的参数,考虑了发射端透镜焦距和孔径的影响。由于在系统的加密发射端对手机相机镜头进行改进,故可使接收系统满足Bokode标签原理。结果表明:该标签系统具有防伪加密功能,能够利用普通手机相机在不低于15cm距离的条件下准确探测和解码。     

自适应柱状全景图拼接

为了实现高效高精度的全景图拼接,提出了一种面向柱状全景图的自适应拼接方法。该方法首先使用基于灰度边缘特征点的图像匹配来获得匹配坐标,并由匹配坐标唯一性计算焦距;然后在该焦距参数估计的基础上,采用新的自适应混合图像灰度匹配方法生成初期拼接图像,并根据其宽度以及相关判别条件进行焦距估计的调整,再重复迭代,以获得准确的焦距参数;最后利用基于角度1维投影的图像匹配方法进行匹配坐标的小范围校正,以及使用线性平滑算法生成最终柱状全景图。该法在精确焦距的基础上,结合了多种图像匹配算法,并根据自相关指数自适应选取结果,实现了水平和垂直两方向上的匹配定位和错位检测。实验证明,该方法不仅在真实图像上的可行,且具有较高的实时性、精度和鲁棒性。