Simultaneously Estimating the Fundamental Matrix and Homographies

The estimation of the fundamental matrix (FM) and/or one or more homographies between two views is of great interest for a number of computer vision and robotics tasks. We consider the joint estimation of the FM and one or more homographies. Given point matches between two views (and assuming rigid geometry of the camera-scene displacement), it is well known that all of the matched points satisfy the epipolar constraint that is usually characterized by the FM. Subsets of these point matches may also obey a constraint characterized by a homography (all matches in the subset coming from three-dimensional (3-D) points lying on a 3-D plane). The estimations of homographies and the FM are well-studied problems, and therefore, the (separate) estimation of the FM, or the homography matrices, can be considered as effectively solved problems with mature algorithms. However, the homographies and FM are not independent of each other: therefore, separate estimation of each is likely to be suboptimal. In this paper, we propose to simultaneously estimate the FM and homographies by employing the compatibility constraint between them. This is done by first concentrating on a set of parameters that (jointly) parameterize the entire set of homographies and FM (simultaneously) and that also implicitly enforce the compatibility between the estimates of each set. We then derive a reduced form with the purpose of improving the speed. We propose a solution method in which the Sampson error for the FM and homographies is minimized by the Levenberg–Marquardt (LM) algorithm. Experiments show that the gains can be compared with separate estimates (the FM and/or the homographies).      

同形映射下的曲面场景的匹配算法的性能分析

针对移动机器人在曲面场景的匹配问题中,同形约束用于解决极约束产生的匹配模糊性问题和发现新的匹配点.实际上是平面块对曲面进行近似逼近的过程.逼近程度和逼近性能需要有指标进行定性和定量的衡量.故提出了两种性能评价指标:平均映射误差和平均映射匹配对.仿真实验结果的分析证明,场景深度变化或者场景距离摄像机的距离变化,对立体匹配算法性能本身不受影响,但映射和建立匹配关系时所需要的同形矩阵的数量不同.而且,随着特征点的稠密度提高,曲面场景的稳定性降低.可随着迭代过程的进行,算法本身结果还趋于稳定.     

ReigSAC: fast discrimination of spurious keypoint correspondences on planar surfaces

Various methods were proposed to detect/match special interest points (keypoints) in images and some of them (e.g., SIFT and SURF) are among the most cited techniques in computer vision research. This paper describes an algorithm to discriminate between genuine and spurious keypoint correspondences on planar surfaces. We draw random samples of the set of correspondences, from which homographies are obtained and their principal eigenvectors extracted. Density estimation on that feature space determines the most likely true transform. Such homography feeds a cost function that gives the goodness of each keypoint correspondence. Being similar to the well-known RANSAC strategy, the key finding is that the main eigenvector of the most (genuine) homographies tends to represent a similar direction. Hence, density estimation in the eigenspace dramatically reduces the number of transforms actually evaluated to obtain reliable estimations. Our experiments were performed on hard image data sets, and pointed that the proposed approach yields effectiveness similar to the RANSAC strategy, at significantly lower computational burden, in terms of the proportion between the number of homographies generated and those that are actually evaluated.     

Multiple homographies with omnidirectional vision for robot homing

Two relevant issues in vision-based navigation are the field-of-view constraints of conventional cameras and the model and structure dependency of standard approaches. A good solution of these problems is the use of the homography model with omnidirectional vision. However, a plane of the scene will cover only a small part of the omnidirectional image, missing relevant information across the wide range field of view, which is the main advantage of omnidirectional sensors. The interest of this paper is in a new approach for computing multiple homographies from virtual planes using omnidirectional images and its application in an omnidirectional vision-based homing control scheme. The multiple homographies are robustly computed, from a set of point matches across two omnidirectional views, using a method that relies on virtual planes independently of the structure of the scene. The method takes advantage of the planar motion constraint of the platform and computes virtual vertical planes from the scene. The family of homographies is also constrained to be embedded in a three-dimensional linear subspace to improve numerical consistency. Simulations and real experiments are provided to evaluate our approach.     

Robust and Efficient Feature Tracking for Indoor Navigation

Robust feature tracking is a requirement for many computer vision tasks such as indoor robot navigation. However, indoor scenes are characterized by poorly localizable features. As a result, indoor feature tracking without artificial markers is challenging and remains an attractive problem. We propose to solve this problem by constraining the locations of a large number of nondistinctive features by several planar homographies which are strategically computed using distinctive features. We experimentally show the need for multiple homographies and propose an illumination-invariant local-optimization scheme for motion refinement. The use of a large number of nondistinctive features within the constraints imposed by planar homographies allows us to gain robustness. Also, the lesser computation cost in estimating these nondistinctive features helps to maintain the efficiency of the proposed method. Our local-optimization scheme produces subpixel accurate feature motion. As a result, we are able to achieve robust and accurate feature tracking.     

基于平面区域跟踪的目标位姿参数自动测量

提出一种从序列图像中自动跟踪测量目标位置和姿态参数的方法。利用单应性原理和上一帧图像中目标位姿参数的测量结果,将目标上的典型平面区域重建为同时含有几何信息和亮度信息的平面区域模板;然后根据投影方程,将该模板在一定的位置姿态参数下进行投影仿真成像,当模板的仿真成像结果与当前帧图像中的该平面区域达到最佳匹配时,认为此时仿真成像的位置姿态参数即为当前帧图像的测量结果。通过对该匹配问题进行最优化建模和求解,实现了序列图像中目标位姿参数的自动测量。实验结果表明,本文方法能够在序列图像中对含有典型平面区域的目标实现较高精度的自动跟踪测量。     

基于SURF与光流法的增强现实跟踪注册

提出一种基于SURF与光流法相结合的增强现实局部跟踪注册方法。采用光流法对移动对象区域进行跟踪,利用SURF算法仿射、尺度不变性及运算速度快的优点对该区域进行特征提取与匹配,利用相邻帧之间特征点的匹配关系求得三维注册矩阵,在保持注册精确性的同时降低了系统运算时间。实验结果表明该方法达到了实时跟踪与准确注册的效果,并且在环境变化时保持了较好的鲁棒性。     

How to put probabilities on homographies

We present a family of "normal" distributions over a matrix group together with a simple method for estimating its parameters. In particular, the mean of a set of elements can be calculated. The approach is applied to planar projective homographies, showing that using priors defined in this way improves object recognition.     

基础矩阵估计的聚类分析算法

提出一种基于聚类分析的Robust基础矩阵估计算法．该算法用高斯混合模型描述匹配点估计余差,采用改进的分裂合并EM算法对匹配点估计余差进行聚类分析,根据分类结果及平均余差最小规则筛选出正确匹配点类别,抛弃错误匹配点;最后,用M估计算法对筛选出的正确匹配点进行迭代求精．大量实验结果表明,文中算法比随机抽样一致性算法的估计精度高,且计算效率高．     

Feature transfer and matching in disparate stereo views through the use of plane homographies

Many vision tasks rely upon the identification of sets of corresponding features among different images. This paper presents a method that, given some corresponding features in two stereo images, matches them with features extracted from a second stereo pair captured from a distant viewpoint. The proposed method is based on the assumption that the viewed scene contains two planar surfaces and exploits geometric constraints that are imposed by the existence of these planes to first transfer and then match image features between the two stereo pairs. The resulting scheme handles point and line features in a unified manner and is capable of successfully matching features extracted from stereo pairs that are acquired from considerably different viewpoints. Experimental results are presented, which demonstrate that the performance of the proposed method compares favorably to that of epipolar and tensor-based approaches.     

Stereo using monocular cues within the tensor voting framework

We address the fundamental problem of matching in two static images. The remaining challenges are related to occlusion and lack of texture. Our approach addresses these difficulties within a perceptual organization framework, considering both binocular and monocular cues. Initially, matching candidates for all pixels are generated by a combination of matching techniques. The matching candidates are then embedded in disparity space, where perceptual organization takes place in 3D neighborhoods and, thus, does not suffer from problems associated with scanline or image neighborhoods. The assumption is that correct matches produce salient, coherent surfaces, while wrong ones do not. Matching candidates that are consistent with the surfaces are kept and grouped into smooth layers. Thus, we achieve surface segmentation based on geometric and not photometric properties. Surface overextensions, which are due to occlusion, can be corrected by removing matches whose projections are not consistent in color with their neighbors of the same surface in both images. Finally, the projections of the refined surfaces on both images are used to obtain disparity hypotheses for unmatched pixels. The final disparities are selected after a second tensor voting stage, during which information is propagated from more reliable pixels to less reliable ones. We present results on widely used benchmark stereo pairs.     

利用平面约束中的反对称性进行相机自定标

回顾了2幅图像中的平面约束，以及一个图像对的基础矩阵和同形矩阵的乘积是一个反对称矩阵的性质，并通过证明展示了这种反对称性质和平面约束之问的关系。给定两幅图像中的一系列对应点，利用反对称性质提出了一种改进的相机自定标算法，将利用平面约束进行相机自定标过程中求取同形矩阵(homography matrix)的问题转化成了方程组约束条件下的二次规划问题，通过解决给定的二次规划问题求解同形矩阵，提高了算法的鲁棒性，然后利用平面约束求解内参数，最后通过本质矩阵(essential matrix)和基础矩阵(fundarnental matrix)之间的关系以及旋转矩阵的性质求解相机外参数。实验结果表明，算法在稳定性方面有了较大程度的提高。     

引入极线约束的SURF特征匹配算法

目的 特征点匹配算法是当今计算机图像处理领域的研究热点,但是大多数现存的方法不能同时获得数量多和质量优的匹配。鉴于此,基于SURF （speeded-up robust features）算法,通过引入极线约束来提高特征匹配效果。方法 首先使用SURF算法检测和描述图像特征点,然后使用RANSAC （random sampling consensus）方法计算匹配图像之间的基础矩阵,通过该基础矩阵计算所有特征点的极线。再引入极线约束过滤掉错误匹配,最终获得数量与质量显著提高的匹配集合。结果 实验结果表明,该方法获得的匹配具有高准确度,匹配数目与原约束条件相比可高达2～8倍。结论 本文方法实现过程简单,不仅匹配准确度高且能够大大提高正确的特征匹配数,适用于处理不同类型的图像数据。     

基于网格对应的双约束特征点匹配算法

常用的特征点匹配算法通常设置严苛的阈值以剔除错误匹配,但这样也会导致过多的正确匹配被删除。针对这一问题,提出了一种采用双约束的特征点匹配方法。首先,在局部上统计特征点匹配数量,运用网格对应的方法过滤部分错误匹配;然后,在全局上运用RANSAC方法计算基础矩阵,通过极线约束对匹配进行再一次筛选。实验表明,相比于传统的匹配算法,该算法能在不增加算法运行时间的前提下,获得更高数量和更高质量的匹配集合。     

Combinatorial Construction of Seamless Parameter Domains

The problem of seamless parametrization of surfaces is of interest in the context of structured quadrilateral mesh generation and spline-based surface approximation. It has been tackled by a variety of approaches, commonly relying on continuous numerical optimization to ultimately obtain suitable parameter domains. We present a general combinatorial seamless parameter domain construction, free from the potential numerical issues inherent to continuous optimization techniques in practice. The domains are constructed as abstract polygonal complexes which can be embedded in a discrete planar grid space, as unions of unit squares. We ensure that the domain structure matches any prescribed parametrization singularities (cones) and satisfies seamlessness conditions. Surfaces of arbitrary genus are supported. Once a domain suitable for a given surface is constructed, a seamless and locally injective parametrization over this domain can be obtained using existing planar disk mapping techniques, making recourse to Tutte's classical embedding theorem.     

多平面多视点单应矩阵间的约束

用代数方法系统地讨论了多平面多视点下单应矩阵间的约束关系.主要结论有(A)如 果视点间摄像机的运动为纯平移运动,则1)对于所有平面关于两视点间的单应矩阵的集合,或 单个平面关于所有视点的单应矩阵的集合的秩均等于4,2)对于多平面多视点的标准单应矩阵 的集合其秩仍等于4,3)根据以上结论可推出现有文献中关于"相对单应矩阵"约束的所有结 果;(B)如果视点间摄像机的运动为一般运动,则1)对于所有平面关于两个视点间的单应矩阵 集合的秩等于4的结论仍成立,2)对于其它情况其秩不再等于4而是等于9.     

Plane-based camera self-calibration by metric rectification of images

Plane-based self-calibration aims at the computation of camera intrinsic parameters from homographies relating multiple views of the same unknown planar scene. This paper proposes a straightforward geometric statement of plane-based self-calibration, through the concept of metric rectification of images. A set of constraints is derived from a decomposition of metric rectification in terms of intrinsic parameters and planar scene orientation. These constraints are then solved using an optimization framework based on the minimization of a geometrically motivated cost function. The link with previous approaches is demonstrated and our method appears to be theoretically equivalent but conceptually simpler. Moreover, a solution dealing with radial distortion is introduced. Experimentally, the method is compared with plane-based calibration and very satisfactory results are obtained. Markerless self-calibration is demonstrated using an intensity-based estimation of the inter-image homographies.     

Visual correction for mobile robot homing

We present a method to send a mobile robot to locations specified by images previously taken from these positions, which sometimes has been referred as homing. Classically this has been carried out using the fundamental matrix, but the fundamental matrix is ill conditioned with planar scenes, which are quite usual in man made environments. Many times in robot homing, small baseline images with high disparity due to rotation are compared, where the fundamental matrix also gives bad results. We use a monocular vision system and we compute motion through an homography obtained from automatically matched lines. In this work we compare the use of the homography and the fundamental matrix and we propose the correction of motion directly from the parameters of the 2D homography, which only needs one calibration parameter. It is shown that it is robust, sufficiently accurate and simple.     

Feature-Based Sequence-to-Sequence Matching

This paper studies the problem of matching two unsynchronized video sequences of the same dynamic scene, recorded by different stationary uncalibrated video cameras. The matching is done both in time and in space, where the spatial matching can be modeled by a homography (for 2D scenarios) or by a fundamental matrix (for 3D scenarios). Our approach is based on matching space-time trajectories of moving objects, in contrast to matching interest points (e.g., corners), as done in regular feature-based image-to-image matching techniques. The sequences are matched in space and time by enforcing consistent matching of all points along corresponding space-time trajectories. By exploiting the dynamic properties of these space-time trajectories, we obtain sub-frame temporal correspondence (synchronization) between the two video sequences. Furthermore, using trajectories rather than feature-points significantly reduces the combinatorial complexity of the spatial point-matching problem when the search space is large. This benefit allows for matching information across sensors in situations which are extremely difficult when only image-to-image matching is used, including: (a) matching under large scale (zoom) differences, (b) very wide base-line matching, and (c) matching across different sensing modalities (e.g., IR and visible-light cameras). We show examples of recovering homographies and fundamental matrices under such conditions.