基于光束法平差的双线阵系统标定技术

针对线阵相机特殊使用场景中所需要的高精度图像,对线阵相机进行高精度标定。提出一种基于光束法平差的双线阵相机标定方法。通过背景差分法获取线阵相机的特征点的像素坐标。再利用已有的直接线性变换方法和非线性优化方法求出相机的内参,外参,畸变参数后,将得到的初始参数与世界坐标作为待优化集合,利用LM法和光束法平差对该集合进行更进一步的优化,使得双线阵系统的重投影误差降到最低。实验表明,该方法与传统的线阵相机标定方法相比重投影误差降低了75.01%。     

Recovering Multiple View Geometry from Mutual Projections of Multiple Cameras

In this paper, we analyze the computation of epipolar geometry in some special cases where multiple cameras are projected each other in their images. In such cases, epipoles can be obtained directly from images as the projection of cameras. As the result, the epipolar geometry can be computed from less image correspondences with higher stability. In this paper, we analyze the number of corresponding points required for computing bifocal, trifocal and quadrifocal tensors linearly in the case where cameras are projected mutually. We next show a practical linear method for computing multifocal tensors by using the mutual projection of cameras. The degenerate configurations of points and cameras is also studied, and it is shown that some degenerate configurations in general cases are no longer degenerate under the mutual projection of cameras.     

Optimizing PTZ camera calibration from two images

In this paper, we address the problem of calibrating an active pan–tilt–zoom (PTZ) camera. In this regard, we make three main contributions: first, for the general camera rotation, we provide a novel solution that yields four independent constraints from only two images, by directly decomposing the infinite homography using a series of Givens rotations. Second, for a camera varying its focal length, we present a solution for the degenerate cases of pure pan and pure tilt that occur very frequently in practical applications of PTZ cameras. Third, we derive a new optimized error function for pure rotation or pan–tilt rotation, which plays a similar role as the epipolar constraint in a freely moving camera, in terms of characterizing the reprojection error of point correspondences. Our solutions and analysis are thoroughly validated and tested on both synthetic and real data, whereby the new geometric error function is shown to outperform existing methods in terms of accuracy and noise resilience.     

Epipolar Geometry and Linear Subspace Methods: A New Approach to Weak Calibration

This paper addresses the problem of estimating the epipolar geometry from point correspondences between two images taken by uncalibrated perspective cameras. It is shown that Jepson's and Heeger's linear subspace technique for infinitesimal motion estimation can be generalized to the finite motion case by choosing an appropriate basis for projective space. This yields a linear method for weak calibration. The proposed algorithm has been implemented and tested on both real and synthetic images, and it is compared to other linear and non-linear approaches to weak calibration.     

Hand–eye calibration with epipolar constraints: Application to endoscopy

In this paper, we propose a new calibration method for a hand–eye system equipped with a camera undergoing radial distortion as a rigid endoscope. While classic methods propose either a separated estimation of the camera intrinsics and the hand–eye transform or a mixed non-linear estimation of both hand–eye and camera intrinsics assuming a pin-hole model, the proposed approach enables a simultaneous refinement of the hand–eye and the camera parameters including the distortion factor with only three frames of the calibrated pattern. We propose and compare two criteria to minimize: (i) the first one is based on the epipolar constraint between pairs of frames and (ii) the second one expresses the camera extrinsic with respect to hand–eye and world-grid transforms in the single frame reprojection error. We run bundle-adjustment on each criterion with respect to the distortion parameters, the camera intrinsics and the hand–eye transform. Our method allows us to recover more accurate 3D shapes when compared to state-of-the-art non-linear methods.     

紧耦合的移动端实时位姿优化方法

位姿估计一直是三维重建领域的关键性问题.为保证移动端有限计算资源下的实时性并提高轨迹计算的准确性,提出一种紧耦合的移动端实时位姿优化方法.首先,获取图像信息与运动传感器信息进行特征提取、预积分等预处理;然后根据对极几何约束,计算重投影误差与惯性传感器误差;最后采用加权误差联合优化计算位姿轨迹.紧耦合策略可以有效利用图像信息与惯性运动信息对位姿约束的一致性.在公开数据集EuRoC上的实验表明,与已有视觉惯性位姿估计方法相比,文中方法在保证移动端实时性的同时,重建相机轨迹误差更小.     

Fundamental matrix estimation: A study of error criteria

The fundamental matrix (FM) describes the geometric relations that exist between two images of the same scene. Different error criteria are used for estimating FMs from an input set of correspondences. In this paper, the accuracy and efficiency aspects of the different error criteria are studied. We mathematically and experimentally proved that the most popular error criterion, the symmetric epipolar distance, is biased. It was also shown that despite the similarity between the algebraic expressions of the symmetric epipolar distance and Sampson distance, they have different accuracy properties. In addition, a new error criterion, Kanatani distance, was proposed and proved to be the most effective for use during the outlier removal phase from accuracy and efficiency perspectives. To thoroughly test the accuracy of the different error criteria, we proposed a randomized algorithm for Reprojection Error-based Correspondence Generation (RE-CG). As input, RE-CG takes an FM and a desired reprojection error value d. As output, RE-CG generates a random correspondence having that error value. Mathematical analysis of this algorithm revealed that the success probability for any given trial is 1 − (2/3)² at best and is 1 − (6/7)² at worst while experiments demonstrated that the algorithm often succeeds after only one trial.     

Affine reconstruction of curved surfaces from uncalibrated views ofapparent contours

In this paper, we consider uncalibrated reconstruction of curved surfaces from apparent contours. Since apparent contours are not fixed features (viewpoint independent), we cannot directly apply the recent results of the uncalibrated reconstruction from fixed features. We show that, nonetheless, curved surfaces can be reconstructed up to an affine ambiguity from their apparent contours viewed from uncalibrated cameras with unknown linear translations. Furthermore, we show that, even if the reconstruction is nonmetric (non-Euclidean), we can still extract useful information for many computer vision applications just from the apparent contours. We first show that if the camera motion is linear translation (but arbitrary direction and magnitude), the epipolar geometry can be recovered from the apparent contours without using any optimization process. The extracted epipolar geometry is next used for reconstructing curved surfaces from the deformations of the apparent contours viewed from uncalibrated cameras. The result is applied to distinguishing curved surfaces from fixed features in images. It is also shown that the time-to-contact to the curved surfaces can be computed from simple measurements of the apparent contours     

Web-Based Remote Renderingwith IBRAC (Image-Based Rendering Acceleration and Compression)

Recent advances in Internet and computer graphics stimulate intensive use and development of 3D graphics on the World Wide Web. To increase efficiency of systems using 3D graphics on the web, the presented method utilizes previously rendered and transmitted images to accelerate the rendering and compression of new synthetic scene images. The algorithm employs ray casting and epipolar constraints to exploit spatial and temporal coherence between the current and previously rendered images. The reprojection of color and visibility data accelerates the computation of new images. The rendering method intrinsically computes a residual image, based on a user specified error tolerance that balances image quality against computation time and bandwidth. Encoding and decoding uses the same algorithm, so the transmitted residual image consists only of significant data without addresses or offsets. We measure rendering speed-ups of four to seven without visible degradation. Compression ratios per frame are a factor of two to ten better than MPEG2 in our test cases. There is no transmission of 3D scene data to delay the first image. The efficiency of the server and client generally increases with scene complexity or data size since the rendering time is predominantly a function of image size. This approach is attractive for remote rendering applications such as web-based scientific visualization where a client system may be a relatively low-performance machine and limited network bandwidth makes transmission of large 3D data impractical.     

Estimating the fundamental matrix via constrained least-squares: aconvex approach

In this paper, a new method for the estimation of the fundamental matrix from point correspondences in stereo vision is presented. The minimization of the algebraic error is performed while taking explicitly into account the rank-two constraint on the fundamental matrix. It is shown how this nonconvex optimization problem can be solved avoiding local minima by using recently developed convexification techniques. The obtained estimate of the fundamental matrix turns out to be more accurate than the one provided by the linear criterion, where the rank constraint of the matrix is imposed after its computation by setting the smallest singular value to zero. This suggests that the proposed estimate can be used to initialize nonlinear criteria, such as the distance to epipolar lines and the gradient criterion, in order to obtain a more accurate estimate of the fundamental matrix     

Pixel History Linear Models for Real‐Time Temporal Filtering

We propose a new real‐time temporal filtering and antialiasing (AA) method for rasterization graphics pipelines. Our method is based on Pixel History Linear Models (PHLM), a new concept for modeling the history of pixel shading values over time using linear models. Based on PHLM, our method can predict per‐pixel variations of the shading function between consecutive frames. This combines temporal reprojection with per‐pixel shading predictions in order to provide temporally coherent shading, even in the presence of very noisy input images. Our method can address both spatial and temporal aliasing problems under a unique filtering framework that minimizes filtering error through a recursive least squares algorithm. We demonstrate our method working with a commercial deferred shading engine for rasterization and with our own OpenGL deferred shading renderer. We have implemented our method in GPU and it has shown significant reduction of temporal flicker in very challenging scenarios including foliage rendering, complex non‐linear camera motions, dynamic lighting, reflections, shadows and fine geometric details. Our approach, based on PHLM, avoids the creation of visible ghosting artifacts and it reduces the filtering overblur characteristic of temporal deflickering methods. At the same time, the results are comparable to state‐of‐the‐art real‐time filters in terms of temporal coherence.     

Achieving invariance to the temporal offset of unsynchronized cameras through epipolar point-line triangulation

This paper addresses the stereo camera synchronization problem for dynamic scenes by proposing a new triangulation method which is invariant to the temporal offset of the cameras. Contrary to spatio-temporal alignment approaches, our method estimates the correct positions of the tracked points without explicitly estimating the temporal offset of the cameras. The method relies on epipolar geometry. In the presence of a temporal delay, a tracked point does not lie on its corresponding epipolar line, thereby inducing triangulation errors. We propose to solve this problem by intersecting its motion trajectory with the corresponding epipolar line. The method we propose does not require calibrated cameras, since it can rely on the fundamental matrix. One major advantage of our approach is that the temporal offset can change throughout the sequence. Evaluated with synthetic and real data, our method proves to be stable and robust to varying temporal offset as well as complex motions.     

一种新的射影重建方法*

提出一种新的射影重建方法。它以最小化射影三维空间点的平均二维反投影误差为准则,以奇异值分解为工具,分步线性迭代实现求解过程,避免了传统射影重建方法中复杂的非线性优化过程和基础矩阵计算过程,且不需要任何初始估计,适用于匹配特征点存在“丢失点”的情况,不受相机特殊运动的限制。利用虚拟物体和真实物体图像序列进行了实验,证明该方法具有计算简单、准确性和鲁棒性高等方面的特点,具有较高的实用价值。     

Motion Recovery by Integrating over the Joint Image Manifold

Recovery of epipolar geometry is a fundamental problem in computer vision. The introduction of the “joint image manifold” (JIM) allows to treat the recovery of camera motion and epipolar geometry as the problem of fitting a manifold to the data measured in a stereo pair. The manifold has a singularity and boundary, therefore special care must be taken when fitting it. Four fitting methods are discussed—direct, algebraic, geometric, and the integrated maximum likelihood (IML) based method. The first three methods are the exact analogues of three common methods for recovering epipolar geometry. The more recently introduced IML method seeks the manifold which has the highest “support,” in the sense that the largest measure of its points are close to the data. While computationally more intensive than the other methods, its results are better in some scenarios. Both simulations and experiments suggest that the advantages of IML manifold fitting carry over to the task of recovering epipolar geometry, especially when the extent of the data and/or the motion are small.     

Outlier correction from uncalibrated image sequence using the Triangulation method

We propose a robust algorithm for estimating the projective reconstruction from image features using the RANSAC-based Triangulation method. In this method, we select input points randomly, separate the input points into inliers and outliers by computing their reprojection error, and correct the outliers so that they can become inliers. The reprojection error and correcting outliers are computed using the Triangulation method. After correcting the outliers, we can reliably recover projective motion and structure using the projective factorization method. Experimental results showed that errors can be reduced significantly compared to the previous research as a result of robustly estimated projective reconstruction.     

Stereo with Oblique Cameras

Mosaics acquired by pushbroom cameras, stereo panoramas, omnivergent mosaics, and spherical mosaics can be viewed as images taken by non-central cameras, i.e. cameras that project along rays that do not all intersect at one point. It has been shown that in order to reduce the correspondence search in mosaics to a one-parametric search along curves, the rays of the non-central cameras have to lie in double ruled epipolar surfaces. In this work, we introduce the oblique stereo geometry, which has non-intersecting double ruled epipolar surfaces. We analyze the configurations of mutually oblique rays that see every point in space. These configurations, called oblique cameras, are the most non-central cameras among all cameras. We formulate the assumption under which two oblique cameras posses oblique stereo geometry and show that the epipolar surfaces are non-intersecting double ruled hyperboloids and two lines. We show that oblique cameras, and the correspondingoblique stereo geometry, exist and give an example of a physically realizable oblique stereo geometry. We introduce linear oblique cameras as those which can be generated by a linear mapping from points in space to camera rays and characterize those collineations which generate them. We show that all linear oblique cameras are obtained by a collineation from one example of an oblique camera. Finally, we relate oblique cameras to spreads known from incidence geometries.     

基于射影空间的视觉基础矩阵鲁棒估计

针对传统视觉基础矩阵计算中存在的噪音和误匹配问题,提出了基于射影空间的基础矩阵计算方法．首先定义了三视几何中射影标准基下基础矩阵只含有5个参数的特殊形式,并利用三视射影重建中空间点反投影图像误差最小为准则,消除了图像中误匹配的影响,然后基于RANSAC(random sampling consensus)技术寻找出最优7个匹配点（噪音最小）来进行对极几何估算．大量仿真模拟试验和真实图像表明此方法能够高精度地估计出基础矩阵．     

基于凸优化改进的相机全局位置估计方法

相机全局位置估计作为运动恢复结构算法（Structure from motion,SfM）中的核心内容一直以来都是计算机视觉领域的研究热点.现有相机全局位置估计方法大多对外点敏感,在处理大规模、无序图像集时表现的尤为明显.增量式SfM中的迭代优化步骤可以剔除大部分的误匹配从而降低外点对估计结果的影响,而全局式SfM中没有有效地剔除误匹配的策略,估计结果受外点影响较大.针对这种情况,本文提出一种改进的相机全局位置估计方法:首先,结合极线约束提出一种新的对误匹配鲁棒的相对平移方向估计算法,减少相对平移方向估计结果中存在的外点;然后,引入平行刚体理论提出一种新的预处理方法将相机全局位置估计转化为一个适定性问题;最后,在此基础上构造了一个对外点鲁棒的凸优化线性估计模型,对模型解算获取相机位置估计全局最优解.本文方法可以很好地融合到当下的全局式SfM流程中.与现有典型方法的对照实验结果表明:在处理大规模、无序图像时,本文方法能显著提高相机全局位置估计的鲁棒性,并保证估计过程的高效性和估计结果的普遍精度.     

一种基于校正误差的立体相机标定算法

立体相机的标定是一个精确求解各个相机内参数以及相机之间关系参数的过程.它是三维重建的基础,其标定精度的好坏直接影响立体重建的结果.为此提出了一种使用校正误差作为代价函数的立体相机标定算法.该算法首先使用传统的基于重投影误差的方法对单个相机的内参数进行标定,然后利用校正误差完成对相机之间关系参数的标定求解.由于校正误差的计算只与相机内参数以及关系参数有关,可以避免在标定过程中使用难以精确标定的相机外参数.实验结果表明本算法能够有效的提高立体相机标定的精度.     

结合SURF算子和极线约束的柑橘立体图像对匹配

提出一种结合SURF算子和极线约束的立体匹配方法。对采集的双目视觉柑橘图像进行R-B分量的计算,在该分量上,采用快速hessian检测子进行特征点检测,并使用SURF描述子对检测到的特征点进行64维的特征描述。采用欧式距离和极线约束进行特征点匹配。实验表明,该方法对一幅图像对的平均处理时间为293ms,在果实被遮挡或光线变化的情况下均能较好地进行特征点提取和匹配。该方法为后续的深度信息计算提供了基础。