Multiview constraints on homographies

The image motion of a planar surface between two camera views is captured by a homography (a 2D projective transformation). The homography depends on the intrinsic and extrinsic camera parameters, as well as on the 3D plane parameters. While camera parameters vary across different views, the plane geometry remains the same. Based on this fact, we derive linear subspace constraints on the relative homographies of multiple (⩾ 2) planes across multiple views. The paper has three main contributions: 1) We show that the collection of all relative homographies (homologies) of a pair of planes across multiple views, spans a 4-dimensional linear subspace. 2) We show how this constraint can be extended to the case of multiple planes across multiple views. 3) We show that, for some restricted cases of camera motion, linear subspace constraints apply also to the set of homographies of a single plane across multiple views. All the results derived are true for uncalibrated cameras. The possible utility of these multiview constraints for improving homography estimation and for detecting nonrigid motions are also discussed     

Multiple homographies with omnidirectional vision for robot homing

Two relevant issues in vision-based navigation are the field-of-view constraints of conventional cameras and the model and structure dependency of standard approaches. A good solution of these problems is the use of the homography model with omnidirectional vision. However, a plane of the scene will cover only a small part of the omnidirectional image, missing relevant information across the wide range field of view, which is the main advantage of omnidirectional sensors. The interest of this paper is in a new approach for computing multiple homographies from virtual planes using omnidirectional images and its application in an omnidirectional vision-based homing control scheme. The multiple homographies are robustly computed, from a set of point matches across two omnidirectional views, using a method that relies on virtual planes independently of the structure of the scene. The method takes advantage of the planar motion constraint of the platform and computes virtual vertical planes from the scene. The family of homographies is also constrained to be embedded in a three-dimensional linear subspace to improve numerical consistency. Simulations and real experiments are provided to evaluate our approach.     

Rank Constraints for Homographies over Two Views: Revisiting the Rank Four Constraint

It is well known that one can collect the coefficients of five (or more) homographies between two views into a large, rank deficient matrix. In principle, this implies that one can refine the accuracy of the estimates of the homography coefficients by exploiting the rank constraint. However, the standard rank-projection approach is impractical for two different reasons: it requires many homographies to even score a modest gain; and, secondly, correlations between the errors in the coefficients will lead to poor estimates. In this paper we study these problems and provide solutions to each. Firstly, we show that the matrices of the homography coefficients can be recast into two parts, each consistent with ranks of only one. This immediately establishes the prospect of realistically (that is, with as few as only three or four homographies) exploiting the redundancies of the homographies over two views. We also tackle the remaining issue: correlated coefficients. We compare our approach with the “gold standard”; that is, non-linear bundle adjustment (initialized from the ground truth estimate—the ideal initialization). The results confirm our theory and show one can implement rank-constrained projection and come close to the gold standard in effectiveness. Indeed, our algorithm (by itself), or our algorithm further refined by a bundle adjustment stage; may be a practical algorithm: providing generally better results than the “standard” DLT (direct linear transformation) algorithm, and even better than the bundle adjustment result with the DLT result as the starting point. Our unoptimized version has roughly the same cost as bundle adjustment and yet can generally produce close to the “gold standard” estimate (as illustrated by comparison with bundle adjustment initialized from the ground truth). Independent of the merits or otherwise of our algorithm, we have illuminated why the naive approach of direct rank-projection is relatively doomed to failure. Moreover, in revealing that there are further rank constraints, not previously known; we have added to the understanding of these issues, and this may pave the way for further improvements.     

From lines to epipoles through planes in two views

This paper addresses the computation of the fundamental matrix between two views, when camera motion and 3D structure are unknown, but planar surfaces can be assumed. We use line features which are automatically matched in two steps. Firstly, with image based parameters, a set of matches are obtained to secondly compute homographies, which allows to reject wrong ones, and to grow good matches in a final stage. The inclusion of projective transformations gives much better results to match features with short computing overload. When two or more planes are observed, different homographies can be computed, segmenting simultaneously the corresponding planar surfaces. These can be used to obtain the fundamental matrix, which gives constraints for the whole scene. The results show that the global process is robust enough, turning out stable and useful to obtain matches and epipolar geometry from lines in man made environments.     

Feature‐based recursive observer design for homography estimation and its application to image stabilization

This paper presents a new algorithm for online estimation of a sequence of homographies applicable to image sequences obtained from robotic vehicles equipped with vision sensors. The approach taken exploits the underlying Special Linear group structure of the set of homographies along with gyroscope measurements and direct point‐feature correspondences between images to develop temporal filter for the homography estimate. Theoretical analysis and experimental results are provided to demonstrate the robustness of the proposed algorithm. The experimental results show excellent performance and robustness even in the case of very fast camera motions (relative to frame rate) and severe occlusions.     

ReigSAC: fast discrimination of spurious keypoint correspondences on planar surfaces

Various methods were proposed to detect/match special interest points (keypoints) in images and some of them (e.g., SIFT and SURF) are among the most cited techniques in computer vision research. This paper describes an algorithm to discriminate between genuine and spurious keypoint correspondences on planar surfaces. We draw random samples of the set of correspondences, from which homographies are obtained and their principal eigenvectors extracted. Density estimation on that feature space determines the most likely true transform. Such homography feeds a cost function that gives the goodness of each keypoint correspondence. Being similar to the well-known RANSAC strategy, the key finding is that the main eigenvector of the most (genuine) homographies tends to represent a similar direction. Hence, density estimation in the eigenspace dramatically reduces the number of transforms actually evaluated to obtain reliable estimations. Our experiments were performed on hard image data sets, and pointed that the proposed approach yields effectiveness similar to the RANSAC strategy, at significantly lower computational burden, in terms of the proportion between the number of homographies generated and those that are actually evaluated.     

Linear two-view multiple plane geometry

In this contribution we present a method for direct linear multiple plane estimation and optimization. The application is the detection of independent motion by background subtraction or temporal differences using image stabilization. Given a set of consistent homographies and a corresponding image segmentation for an image pair, it is possible to improve the image stabilization by locally warping. We show that linear constraints can be induced to homography estimation to ensure consistency. In this context consistency means that the homographies correspond to the same relative orientation but to different 3D planes. The relative orientation can be derived by measurements of additional sensors, e.g., inertial navigation system, odometry, or can be computed from the fundamental or essential matrix. The method is compared to standard estimation of homographies with simulated data, and the capability of the method is shown using depth estimation on the Middlebury-Stereo Benchmark dataset. The text was submitted by the author in English. Michael Kirchhof (born in 1978 in Herborn, Germany) graduated with a diploma in mathematics from the Justus Liebig University Giessen in 2004. Since 2004 he has been working as scientist in the FGAN-FOM in Ettlingen, Germany. At present he is also working on his PhD thesis at the Bundeswehr University Munich. He is author and coauthor of more than 10 publications. The scope of his scientific interests includes computer vision, image processing, and close range photogrammetry.     

基于随机抽样一致性的多平面区域检测算法

在随机抽样一致性(RANSAC)的基础上,提出了一种对多个平面区域同时进行检测的算法.该算法假设对同一场景的一对未定标图像已经进行了特征点提取和匹配,首先利用对极几何约束计算出一对极点,然后随机抽取多组3对而非4对特征点定义多个待确定单应性矩阵模型,对图像对中的多个平面区域同时进行检测.模拟实验和真实实验都证明该算法具有运算量小、准确性高、鲁棒性好等优点.     

场景无关约束下的特征匹配算法

鉴于对极约束是立体图像中完全不依赖于场景的重要几何约束，因此在特征匹配中起着很重要的作用，而且由于同形映射描述了平面场景的立体图像之间的对应关系，故大量文献中利用它对平面场景的立体图像对进行特征匹配。为了提高立体图像匹配精度和速度，提出了一种改进的场景无关约束下的特征匹配算法，该算法针对用对极约束和同形映射来进行曲面场景匹配的过程中同形估计容易出现降阶的情况，通过引入区域面积检测法来避免降阶情况的发生，以改善匹配结果；同时，由于在同形矩阵估计中，通过加入基础矩阵和同形矩阵本质上的约束关系，可使得原本独立的同形约束和对极约柬关系很好地融入到匹配的整个过程中，从而快速有效地抑制了错误匹配的发生。对真实图像的实验分析证明，该改进算法具有迭代次数少、速度更快和匹配精度高的良好性能。     

Robust multi-view feature matching from multiple unordered views

This paper explores the problem of multi-view feature matching from an unordered set of widely separated views. A set of local invariant features is extracted independently from each view. First we propose a new view-ordering algorithm that organizes all the unordered views into clusters of related (i.e. the same scene) views by efficiently computing the view-similarity values of all view pairs by reasonably selecting part of extracted features to match. Second a robust two-view matching algorithm is developed to find initial matches, then detect the outliers and finally incrementally find more reliable feature matches under the epipolar constraint between two views from dense to sparse based on an assumption that changes of both motion and feature characteristics of one match are consistent with those of neighbors. Third we establish the reliable multi-view matches across related views by reconstructing missing matches in a neighboring triple of views and efficiently determining the states of matches between view pairs. Finally, the reliable multi-view matches thus obtained are used to automatically track all the views by using a self-calibration method. The proposed methods were tested on several sets of real images. Experimental results show that it is efficient and can track a large set of multi-view feature matches across multiple widely separated views.     

A rapid method for detecting objects with rectangular structures based on line correspondences

Rectangles commonly appear in man-made environments. This paper presents an approach for rapid detection of objects with rectangular structures. Considering that the object to be detected may have different textures, colors or other characteristics, only geometric structure information can be used for identification. Therefore, the key task in detecting rectangles is to acquire correct line correspondences. We analyze the reason why a basic frame of 2D homography computation and verification cannot meet the practical requirement for detection speed and then propose a two-step algorithm to solve the problem. First, we utilize a unified expression of homography between an unknown size rectangle and a quadrangle on the image to simplify the homography computation. Second, we propose a rotated cuboid bound constraint of the camera centre to rapidly remove incorrect homographies with unknown size parameters. Experiments with simulated and real data verify the correctness of our method.     

The Geometry and Matching of Lines and Curves Over Multiple Views

This paper describes the geometry of imaged curves in two and three views. Multi-view relationships are developed for lines, conics and non-algebraic curves. The new relationships focus on determining the plane of the curve in a projective reconstruction, and in particular using the homography induced by this plane for transfer from one image to another. It is shown that given the fundamental matrix between two views, and images of the curve in each view, then the plane of a conic may be determined up to a two fold ambiguity, but local curvature of a curve uniquely determines the plane. It is then shown that given the trifocal tensor between three views, this plane defines a homography map which may be used to transfer a conic or the curvature from two views to a third. Simple expressions are developed for the plane and homography in each case.A set of algorithms are then described for automatically matching individual line segments and curves between images. The algorithms use both photometric information and the multiple view geometric relationships. For image pairs the homography facilitates the computation of a neighbourhood cross-correlation based matching score for putative line/curve correspondences. For image triplets cross-correlation matching scores are used in conjunction with line/curve transfer based on the trifocal geometry to disambiguate matches. Algorithms are developed for both short and wide baselines. The algorithms are robust to deficiencies in the segment extraction and partial occlusion.Experimental results are given for image pairs and triplets, for varying motions between views, and for different scene types. The methods are applicable to line/curve matching in stereo and trinocular rigs, and as a starting point for line/curve matching through monocular image sequences.     

Visual motion ambiguities of a plane in 2-D FS sonar motion sequences

Sonar is the most common imaging modality in underwater, and high-resolution high data rate 2-D video systems have been emerging in recent years. As for visually guided terrestrial robot navigation and target-based positioning, the estimation of 3-D motion by tracking features in recorded 2-D sonar images is also a highly desirable capability for submersible platforms. Additionally, theoretical results dealing with robustness and multiplicity of solution constitute important fundamental findings due to nature of sonar data, namely, high noise level, narrow field of view coverage, scarcity of robust features, and incorrect matches.This paper explores the inherent ambiguities of 3-D motion and scene structure interpretation from 2-D forward-scan sonar image sequences. Analyzing the sonar image motion transformation model, which depends on the affine components of the projective transformation (or homography) of two plane views, we show that two interpretations are commonly inferred. The true and spurious planes form mirror images relative to the zero-elevation plane of the sonar reference frame. Even under each of pure rotation or translation, a spurious motion exists comprising both translational and rotational components. In some cases, the two solutions share certain motion components, where the imaged surface becomes parallel to a plane defined by two of the sonar coordinate axes. A unique solution exists under the very special condition where the sonar motion aligns the imaged plane with the zero-elevation planes. We also derive the relationship between the two interpretations, thus allowing closed-form computation of both solutions.     

Dense mirroring surface recovery from 1D homographies and sparse correspondences

In this work, we recover the 3D shape of mirrors, sunglasses, and stainless steel implements. A computer monitor displays several images of parallel stripes, each image at a different angle. Reflections of these stripes in a mirroring surface are captured by the camera. For every image point, the direction of the displayed stripes and their reflections in the image are related by a 1D homography matrix, computed with a robust version of the statistically accurate heteroscedastic approach. By focusing on a sparse set of image points for which monitor-image correspondence is computed, the depth and the local shape may be estimated from these homographies. The depth estimation relies on statistically correct minimization and provides accurate, reliable results. Even for the image points where the depth estimation process is inherently unstable, we are able to characterize this instability and develop an algorithm to detect and correct it. After correcting the instability, dense surface recovery of mirroring objects is performed using constrained interpolation, which does not simply interpolate the surface depth values but also uses the locally computed 1D homographies to solve for the depth, the correspondence, and the local surface shape. The method was implemented and the shape of several objects was densely recovered at submillimeter accuracy.     

同形映射下的曲面场景的匹配算法的性能分析

针对移动机器人在曲面场景的匹配问题中,同形约束用于解决极约束产生的匹配模糊性问题和发现新的匹配点.实际上是平面块对曲面进行近似逼近的过程.逼近程度和逼近性能需要有指标进行定性和定量的衡量.故提出了两种性能评价指标:平均映射误差和平均映射匹配对.仿真实验结果的分析证明,场景深度变化或者场景距离摄像机的距离变化,对立体匹配算法性能本身不受影响,但映射和建立匹配关系时所需要的同形矩阵的数量不同.而且,随着特征点的稠密度提高,曲面场景的稳定性降低.可随着迭代过程的进行,算法本身结果还趋于稳定.     

Nonlinear complementary filters on the special linear group

This article proposes a nonlinear complementary filter for the special linear Lie-group SL(3) that fuses low-frequency state measurements with partial velocity measurements and adaptive estimation of unmeasured slowly changing velocity components. The obtained results have direct application on the problem of filtering a sequence of image homographies acquired from low-quality video data. The considered application motivates us to derive results that provide adaptive estimation of the full group velocity or part of the group velocity that cannot be measured from sensors attached to the camera. We demonstrate the performance of the proposed filters on real world homography data.     

A New Analytical Method for Relative Camera Pose Estimation Using Unknown Coplanar Points

We present a new analytical method for solving the problem of relative camera pose estimation. This method first calculates the homography matrix between two calibrated views using unknown coplanar points, and then, it decomposes the matrix to estimate the relative camera pose. We derive a set of new analytical expressions that are more concise than other homography decomposition methods. These analytical expressions are also used to improve the efficiency of a traditional SVD-based homography decomposition method. The performance of our analytical method is studied in terms of both efficiency and accuracy, and it is compared with other homography decomposition methods. Furthermore, the accuracy of our analytical method is tested under different conditions and compared with that of the five-point method through simulations and real image experiments. The experimental results demonstrate that our method is faster and more accurate than other homography decomposition methods and more accurate than the five-point method.     

1D camera geometry and its application to the self-calibration of circular motion sequences

This paper proposes a novel method for robustly recovering the camera geometry of an uncalibrated image sequence taken under circular motion. Under circular motion, all the camera centers lie on a circle and the mapping from the plane containing this circle to the horizon line observed in the image can be modelled as a 1D projection. A 2x2 homography is introduced in this paper to relate the projections of the camera centers in two 1D views. It is shown that the two imaged circular points of the motion plane and the rotation angle between the two views can be derived directly from such a homography. This way of recovering the imaged circular points and rotation angles is intrinsically a multiple view approach, as all the sequence geometry embedded in the epipoles is exploited in the estimation of the homography for each view pair. This results in a more robust method compared to those computing the rotation angles using adjacent views only. The proposed method has been applied to self-calibrate turntable sequences using either point features or silhouettes, and highly accurate results have been achieved.     

On optimally combining pieces of information, with application to estimating 3-d complex-object position from range data

New asymptotic methods are introduced that permit computationally simple Bayesian recognition and parameter estimation for many large data sets described by a combination of algebraic, geometric, and probabilistic models. The techniques introduced permit controlled decomposition of a large problem into small problems for separate parallel processing where maximum likelihood estimation or Bayesian estimation or recognition can be realized locally. These results can be combined to arrive at globally optimum estimation or recognition. The approach is applied to the maximum likelihood estimation of 3-D complex-object position. To this end, the surface of an object is modeled as a collection of patches of primitive quadrics, i.e., planar, cylindrical, and spherical patches, possibly augmented by boundary segments. The primitive surface-patch models are specified by geometric parameters, reflecting location, orientation, and dimension information. The object-position estimation is based on sets of range data points, each set associated with an object primitive. Probability density functions are introduced that model the generation of range measurement points. This entails the formulation of a noise mechanism in three-space accounting for inaccuracies in the 3-D measurements and possibly for inaccuracies in the 3-D modeling. We develop the necessary techniques for optimal local parameter estimation and primitive boundary or surface type recognition for each small patch of data, and then optimal combining of these inaccurate locally derived parameter estimates in order to arrive at roughly globally optimum object-position estimation.     

Detecting motion regions in presence of strong parallax from a moving camera by multi-view geometric constraints

We present a method for detecting motion regions in video sequences observed by a moving camera in the presence of a strong parallax due to static 3D structures. The proposed method classifies each image pixel into planar background, parallax, or motion regions by sequentially applying 2D planar homographies, the epipolar constraint, and a novel geometric constraint called the "structure consistency constraint." The structure consistency constraint, being the main contribution of this paper, is derived from the relative camera poses in three consecutive frames and is implemented within the "Plane + Parallax" framework. Unlike previous planar-parallax constraints proposed in the literature, the structure consistency constraint does not require the reference plane to be constant across multiple views. It directly measures the inconsistency between the projective structures from the same point under camera motion and reference plane change. The structure consistency constraint is capable of detecting moving objects followed by a moving camera in the same direction, a so-called degenerate configuration where the epipolar constraint fails. We demonstrate the effectiveness and robustness of our method with experimental results of real-world video sequences.