True multi-image alignment and its application to mosaicing andlens distortion correction

Multiple images of a scene are related through 2D/3D view transformations and linear and nonlinear camera transformations. We present an algorithm for true multi-image alignment that does not rely on the measurements of a reference image being distortion free. The algorithm is developed to specifically align and mosaic images using parametric transformations in the presence of lens distortion. When lens distortion is present, none of the images can be assumed to be ideal. In our formulation, all the images are modeled as intensity measurements represented in their respective coordinate systems, each of which is related to an ideal coordinate system through an interior camera transformation and an exterior view transformation. The goal of the accompanying algorithm is to compute an image in the ideal coordinate system while solving for the transformations that relate the ideal system with each of the data images. Key advantages of the technique presented in this paper are: (i) no reliance on one distortion free image, (ii) ability to register images and compute coordinate transformations even when the multiple images are of an extended scene with no overlap between the first and last frame of the sequence, and (iii) ability to handle linear and nonlinear transformations within the same framework. Results of applying the algorithm are presented for the correction of lens distortion, and creation of video mosaics     

Accurate Depth Dependent Lens Distortion Models: An Application to Planar View Scenarios

In order to calibrate cameras in an accurate manner, lens distortion models have to be included in the calibration procedure. Usually, the lens distortion models used in camera calibration depend on radial functions of image pixel coordinates. Such models are well-known, simple and can be estimated using just image information. However, these models do not take into account an important physical constraint of lens distortion phenomena, namely: the amount of lens distortion induced in an image point depends on the scene point depth with respect to the camera projection plane. In this paper we propose a new accurate depth dependent lens distortion model. To validate this approach, we apply the new lens distortion model to camera calibration in planar view scenarios (that is 3D scenarios where the objects of interest lie on a plane). We present promising experimental results on planar pattern images and on sport event scenarios. Nevertheless, although we emphasize the feasibility of the method for planar view scenarios, the proposed model is valid in general and can be used in any scenario where the point depth can be estimated.     

双线阵CCD相机的畸变校正和标定方法

在双线阵CCD的三维重建中,对线阵CCD相机的标定和镜头畸变校正是基础环节。提出了一种用于三维重建中的双线阵CCD标定及镜头畸变校正方法。根据左右相机间的单应性关系,以及线阵CCD的成像原理,将双目相机间的空间关系分解成姿态角与错切角的关系。通过靶图数据的拟合,对姿态角和镜头畸变进行校正,根据求出的错切角完成相机间的标定,实现对具有镜头畸变的双线阵CCD的标定。实验结果表明,标定和校正精度满足后续三维重建中图像匹配的需求。     

基于直线的几何不变性标定摄像机参数

计算机视觉通常采用针孔摄像机模型,但对于存在较大畸变的鱼眼镜头或广角镜头来说,会造成图像中同时存在透视变形和像差畸变。解决此问题的传统方法一般是采用标准网格板来标定摄像机参数,但需要较多的已知信息。为了进行精确的标定,提出了一种新的标定方法,该新方法不需要任何空间3维信息,即可用单幅普通图像来标定摄像机的像差系数及内参数,并可将畸变图像校正到相似变换。为了纠正像差畸变和计算消影点,该方法采用了直线的射影不变性,即共线点的投影仍然共线,平行直线束的投影相交于一点的性质;为了纠正透视变形,还采用了直线的相似不变性,即正交直线的夹角在相似变换中仍然保持正交的性质。用该方法标定的摄像机的参数包括像差系数、焦距、主点和纵横比,同时将图像纠正到了相似变换。用实验室图像和室外图像进行了仿真实验都得到了精确、可靠的结果。     

一种基于斜率的摄像机畸变校正方法

普通 CCD摄像机在成像时都存在畸变成像误差 ,在机器人视觉检测及自动装配中 ,有效地进行误差校正对准确确定物体的位置具有重要的意义 .本文采用带有一阶径向畸变的小孔摄像机模型 ,提出一种基于线段斜率的方法 ,对摄像机镜头的径向畸变进行校正 ,不必标定太多的摄像机的外参数 ,方法简洁 ,适合于视觉系统中对摄像机畸变的实时校正 ,或对摄像机捕获的图像进行几何校正 .实验表明 ,具有很强的鲁棒性和较高的校正精度     

基于直线投影特征的镜头畸变校正方法

针对摄像机镜头畸变校正方法的简便和快速问题,设计了基于直线投影特征的校正方法。介绍了镜头主要畸变产生原因和畸变模型;给出直线三点在理想投影下的关系,确定了适应度函数,利用遗传算法得到了畸变参数组最优解。基于matlab软件编写校正程序,并进行了实验验证。实验表明,利用畸变参数组的最优解能够实现图像畸变校正,效果较好。该标定方法只需场景内有直线存在即可实现对摄像机镜头畸变参数校正,方法所需实验条件简单,程序简便,便于现场快速校正。     

Robust metric calibration of non-linear camera lens distortion

Camera lens distortion is crucial to obtain the best performance cameral model. Up to now, different techniques exist, which try to minimize the calibration error using different lens distortion models or computing them in different ways. Some compute lens distortion camera parameters in the camera calibration process together with the intrinsic and extrinsic ones. Others isolate the lens distortion calibration without using any template and basing the calibration on the deformation in the image of some features of the objects in the scene, like straight lines or circles. These lens distortion techniques which do not use any calibration template can be unstable if a complete camera lens distortion model is computed. They are named non-metric calibration or self-calibration methods.Traditionally a camera has been always best calibrated if metric calibration is done instead of self-calibration. This paper proposes a metric calibration technique which computes the camera lens distortion isolated from the camera calibration process under stable conditions, independently of the computed lens distortion model or the number of parameters. To make it easier to resolve, this metric technique uses the same calibration template that will be used afterwards for the calibration process. Therefore, the best performance of the camera lens distortion calibration process is achieved, which is transferred directly to the camera calibration process.     

由灭点进行径向畸变的自动校正

目的镜头畸变影响着3维重建、几何量测等工作的质量。根据图像中的灭点几何约束条件,提出一种根据灭点进行弱径向畸变自动校正方法。方法为避免包含畸变中心参数的非线性模型优化结果的不稳定性,在求解径向畸变系数之后进一步对畸变中心进行优化。首先根据灭点几何约束条件建立关于灭点与径向畸变系数的非线性模型,使用Levenberg-Marquardt(LM)算法估计灭点坐标与径向畸变参数,然后根据质量评价准则对畸变中心和径向畸变系数进一步迭代优化;最后,通过真实图像对该方法的可行性进行分析验证。结果采用不同的数据对径向畸变进行了有效地校正,并采用校正后的数据进行了相机标定,标定结果相对于传统基于非量测校正方法有明显的提高。结论充分利用图像中的灭点属性,提出一种新的镜头径向畸变校正方法。实验结果表明,该方法能够有效地对径向畸变进行校正,并克服了传统基于非量测畸变校正方法的不稳定性。     

Restoring 2D content from distorted documents

This paper presents a framework to restore the 2D content printed on documents in the presence of geometric distortion and non-uniform illumination. Compared with textbased document imaging approaches that correct distortion to a level necessary to obtain sufficiently readable text or to facilitate optical character recognition (OCR), our work targets nontextual documents where the original printed content is desired. To achieve this goal, our framework acquires a 3D scan of the document's surface together with a high-resolution image. Conformal mapping is used to rectify geometric distortion by mapping the 3D surface back to a plane while minimizing angular distortion. This conformal "deskewing" assumes no parametric model of the document's surface and is suitable for arbitrary distortions. Illumination correction is performed by using the 3D shape to distinguish content gradient edges from illumination gradient edges in the high-resolution image. Integration is performed using only the content edges to obtain a reflectance image with significantly less illumination artifacts. This approach makes no assumptions about light sources and their positions. The results from the geometric and photometric correction are combined to produce the final output.     

Learning Style Compatibility Between Objects in a Real‐World 3D Asset Database

Large 3D asset databases are critical for designing virtual worlds, and using them effectively requires techniques for efficient querying and navigation. One important form of query is search by style compatibility: given a query object, find others that would be visually compatible if used in the same scene. In this paper, we present a scalable, learning‐based approach for solving this problem which is designed for use with real‐world 3D asset databases; we conduct experiments on 121 3D asset packages containing around 4000 3D objects from the Unity Asset Store. By leveraging the structure of the object packages, we introduce a technique to synthesize training labels for metric learning that work as well as human labels. These labels can grow exponentially with the number of objects, allowing our approach to scale to large real‐world 3D asset databases without the need for expensive human training labels. We use these synthetic training labels in a metric learning model that analyzes the in‐engine rendered appearance of an object—combining geometry, material, and texture—whereas prior work considers only object geometry, or disjoint geometry and texture features. Through an ablation experiment, we find that using this representation yields better results than using renders which lack texture, materiality, or both.     

基于混合模型的CCD镜头畸变精校正算法

针对机器视觉检测和高精度图像测量中使用的CCD镜头都存在不同程度光学畸变的问题,提出基于混合模型的CCD镜头畸变校正算法。用经典模型对畸变图像进行初次校正,用多面函数拟合法进行二次精校正,用三次B样条函数进行灰度重建。实验结果表明,该方法在不依赖摄像机内部参数条件下,相比单一的镜头畸变校正模型,精度提高,鲁棒性增强,校正后径向均方根误差为0.3个像素。     

机器视觉系统中图像误差补偿方法

在分析和比较常用图像畸变校正算法的基础上,提出将BP神经网络原理应用于图像的畸变校正,从而在对图像进行检测和定量分析之前,消除畸变误差对测量结果的影响;利用在标准栅格样板图像上选择的控制点对神经网络进行训练,并将训练后的神经网络以动态链接库的形式提供给LabVIEW调用,在LabVIEW环境中实现了具有图像误差补偿功能的机器视觉测量系统.     

基于标志点注册及探测球的光源追踪算法

增强现实系统中,解决虚拟物体表面光照情况与真实环境匹配问题关键的一步是进行光源追踪。基于标志点注册及具有漫反射特性的探测球提出一种光源追踪算法。该算法仅对一幅被单一光源照射的标志立方体和探测球图像进行分析,利用图中标志点确定探测球相对照相机的位置、姿态关系,利用探测球表面的亮度信息推算光源向量。该探测球图像要进行一系列图像处理程序,其中等亮度线的提取及拟合是关键步骤。实验结果表明,本文算法能够达到预期效果,较好地实现了光源追踪,适用于各种位置的单一光源照射情况及基于标志点注册的增强现实系统。     

Confocal Stereo

We present confocal stereo, a new method for computing 3D shape by controlling the focus and aperture of a lens. The method is specifically designed for reconstructing scenes with high geometric complexity or fine-scale texture. To achieve this, we introduce the confocal constancy property, which states that as the lens aperture varies, the pixel intensity of a visible in-focus scene point will vary in a scene-independent way, that can be predicted by prior radiometric lens calibration. The only requirement is that incoming radiance within the cone subtended by the largest aperture is nearly constant. First, we develop a detailed lens model that factors out the distortions in high resolution SLR cameras (12MP or more) with large-aperture lenses (e.g., f1.2). This allows us to assemble an A×F aperture-focus image (AFI) for each pixel, that collects the undistorted measurements over all A apertures and F focus settings. In the AFI representation, confocal constancy reduces to color comparisons within regions of the AFI, and leads to focus metrics that can be evaluated separately for each pixel. We propose two such metrics and present initial reconstruction results for complex scenes, as well as for a scene with known ground-truth shape. Part of this work was done while the authors were visiting Microsoft Research Asia, in the roles of research intern and Visiting Scholar respectively.     

Camera calibration in sport event scenarios

The main goal of this paper is the design of a novel and robust methodology for calibrating cameras from a single image in sport scenarios, such as a soccer field, or a basketball or tennis court. In these sport scenarios, the only references we use to calibrate the camera are the lines and circles delimiting the different regions. The first problem we address is the extraction of image primitives, including the challenging problems of shaded regions and lens distortion. From these primitives, we automatically recognise the location of the sport court in the scene by estimating the homography which matches the actual court with its projection onto the image. This is achieved even when only a few primitives are available. Finally, from this homography, we recover the camera calibration parameters. In particular, we estimate the focal length as well as the position and orientation in the 3D space. We present some experiments on models and real courts which illustrate the accuracy of the proposed methodology.     

一种基于基本矩阵的相机畸变差自动校正方法

在计算机视觉的应用领域中,为了提高图像量测和3维重建的精度,必须对相机的畸变误差进行修正。为此提出了一种基于基本矩阵的相机径向畸变的自动校正方法,该方法不需要预先获得场景的结构信息和相机的内部参数,仅利用两张影像同名点集之间的内在几何关系,即可求取相机的径向畸变系数,进而可对这两幅图像的畸变误差进行自动校正。试验结果表明,该方法是一种有效的畸变图像校正算法,能够获得到满意的校正结果。     

基于直线约束的非量测数码相机标定

提出一种非量测数码相机标定的直接方法。该方法基于直线约束条件,即在正确进行畸变纠正后,物方空间的三维直线投影到像片平面上也应该是一条直线。为了将畸变直线纠正成理想直线,利用畸变模型对其进行畸变纠正,并采用Levenberg-Marquardt算法对非线性方程组进行求解,解出符合条件的最佳畸变参数。实验结果证明该方法具有较强的鲁棒性与实用性。     

Structure from Motion: Beyond the Epipolar Constraint

The classic approach to structure from motion entails a clear separation between motion estimation and structure estimation and between two-dimensional (2D) and three-dimensional (3D) information. For the recovery of the rigid transformation between different views only 2D image measurements are used. To have available enough information, most existing techniques are based on the intermediate computation of optical flow which, however, poses a problem at the locations of depth discontinuities. If we knew where depth discontinuities were, we could (using a multitude of approaches based on smoothness constraints) accurately estimate flow values for image patches corresponding to smooth scene patches; but to know the discontinuities requires solving the structure from motion problem first. This paper introduces a novel approach to structure from motion which addresses the processes of smoothing, 3D motion and structure estimation in a synergistic manner. It provides an algorithm for estimating the transformation between two views obtained by either a calibrated or uncalibrated camera. The results of the estimation are then utilized to perform a reconstruction of the scene from a short sequence of images.The technique is based on constraints on image derivatives which involve the 3D motion and shape of the scene, leading to a geometric and statistical estimation problem. The interaction between 3D motion and shape allows us to estimate the 3D motion while at the same time segmenting the scene. If we use a wrong 3D motion estimate to compute depth, we obtain a distorted version of the depth function. The distortion, however, is such that the worse the motion estimate, the more likely we are to obtain depth estimates that vary locally more than the correct ones. Since local variability of depth is due either to the existence of a discontinuity or to a wrong 3D motion estimate, being able to differentiate between these two cases provides the correct motion, which yields the least varying estimated depth as well as the image locations of scene discontinuities. We analyze the new constraints, show their relationship to the minimization of the epipolar constraint, and present experimental results using real image sequences that indicate the robustness of the method.     

基于视点纠正的鱼眼图像场景化漫游方法

针对鱼眼图像畸变大、表示的场景信息不直观等特点,本文提出了一种 基于视点纠正的鱼眼图像场景化漫游方法。该方法以鱼眼图像所表示的半球空间为观察对 象,通过视点的转移在半球空间进行漫游,以对不同区域内的场景信息进行直观的观察。采 用鱼眼图像校正算法为基础建立鱼眼镜头的球面映射模型,通过映射关系对以视点为中心的 观察区域进行实时校正,当视点变化时对校正后的可视区域进行实时显示从而实现漫游。实 验结果表明,通过本文算法能够实现鱼眼图像所表示半球空间的实时漫游,且漫游时显示的 校正图像满足直线约束标准。     

基于灭点的径向畸变自动校正方法

镜头畸变影响着三维重建、几何量测等工作的质量。本文根据图像中的灭点几何约束条件,提出一种基于灭点的弱径向畸变自动校正方法。为避免包含畸变中心参数的非线性模型优化结果的不稳定性,本文在求解径向畸变系数之后进一步对畸变中心进行优化。首先根据灭点几何约束条件建立关于灭点与径向畸变系数的非线性模型,使用LM算法估计灭点坐标与径向畸变参数,然后根据质量评价准则对畸变中心和径向畸变系数进一步迭代优化;最后,通过真实图像对该方法的可行性进行分析验证。本文充分利用图像中的灭点属性,提出一种新的镜头径向畸变校正方法。实验结果表明,该方法能够有效的对径向畸变进行校正,并克服了传统校正方法的不稳定性。