Simultaneously Estimating the Fundamental Matrix and Homographies

The estimation of the fundamental matrix (FM) and/or one or more homographies between two views is of great interest for a number of computer vision and robotics tasks. We consider the joint estimation of the FM and one or more homographies. Given point matches between two views (and assuming rigid geometry of the camera-scene displacement), it is well known that all of the matched points satisfy the epipolar constraint that is usually characterized by the FM. Subsets of these point matches may also obey a constraint characterized by a homography (all matches in the subset coming from three-dimensional (3-D) points lying on a 3-D plane). The estimations of homographies and the FM are well-studied problems, and therefore, the (separate) estimation of the FM, or the homography matrices, can be considered as effectively solved problems with mature algorithms. However, the homographies and FM are not independent of each other: therefore, separate estimation of each is likely to be suboptimal. In this paper, we propose to simultaneously estimate the FM and homographies by employing the compatibility constraint between them. This is done by first concentrating on a set of parameters that (jointly) parameterize the entire set of homographies and FM (simultaneously) and that also implicitly enforce the compatibility between the estimates of each set. We then derive a reduced form with the purpose of improving the speed. We propose a solution method in which the Sampson error for the FM and homographies is minimized by the Levenberg–Marquardt (LM) algorithm. Experiments show that the gains can be compared with separate estimates (the FM and/or the homographies).      

Nonlinear complementary filters on the special linear group

This article proposes a nonlinear complementary filter for the special linear Lie-group SL(3) that fuses low-frequency state measurements with partial velocity measurements and adaptive estimation of unmeasured slowly changing velocity components. The obtained results have direct application on the problem of filtering a sequence of image homographies acquired from low-quality video data. The considered application motivates us to derive results that provide adaptive estimation of the full group velocity or part of the group velocity that cannot be measured from sensors attached to the camera. We demonstrate the performance of the proposed filters on real world homography data.     

多平面多视点单应矩阵间的约束

用代数方法系统地讨论了多平面多视点下单应矩阵间的约束关系.主要结论有(A)如 果视点间摄像机的运动为纯平移运动,则1)对于所有平面关于两视点间的单应矩阵的集合,或 单个平面关于所有视点的单应矩阵的集合的秩均等于4,2)对于多平面多视点的标准单应矩阵 的集合其秩仍等于4,3)根据以上结论可推出现有文献中关于"相对单应矩阵"约束的所有结 果;(B)如果视点间摄像机的运动为一般运动,则1)对于所有平面关于两个视点间的单应矩阵 集合的秩等于4的结论仍成立,2)对于其它情况其秩不再等于4而是等于9.     

基于多相机的多目标跟踪算法

多目标的稳定跟踪是计算机视觉领域的一个具有挑战性的问题. 本文提出了一种基于多相机的多目标定位跟踪算法.首先, 利用不同高度层上的标志物, 计算基于多层的不同视角间的单应性矩阵.然后, 利用码本模型对背景进行建模, 检测多个视角的前景似然信息.最后, 通过单应性变换获得多目标在不同高度层上的定位信息, 利用最短路径优化算法实现跟踪. 与其他算法相比, 本算法不需要计算多相机的隐消点, 降低了算法的复杂度, 提高了算法的准确性.采用最短路径优化算法, 提高了跟踪算法的效率. 实验结果表明, 本算法对遮挡具有很强的鲁棒性, 并且能够满足实时性要求.     

Registration Based Non‐uniform Motion Deblurring

This paper proposes an algorithm which uses image registration to estimate a non‐uniform motion blur point spread function (PSF) caused by camera shake. Our study is based on a motion blur model which models blur effects of camera shakes using a set of planar perspective projections (i.e., homographies). This representation can fully describe motions of camera shakes in 3D which cause non‐uniform motion blurs. We transform the non‐uniform PSF estimation problem into a set of image registration problems which estimate homographies of the motion blur model one‐by‐one through the Lucas‐Kanade algorithm. We demonstrate the performance of our algorithm using both synthetic and real world examples. We also discuss the effectiveness and limitations of our algorithm for non‐uniform deblurring.     

Rank Constraints for Homographies over Two Views: Revisiting the Rank Four Constraint

It is well known that one can collect the coefficients of five (or more) homographies between two views into a large, rank deficient matrix. In principle, this implies that one can refine the accuracy of the estimates of the homography coefficients by exploiting the rank constraint. However, the standard rank-projection approach is impractical for two different reasons: it requires many homographies to even score a modest gain; and, secondly, correlations between the errors in the coefficients will lead to poor estimates. In this paper we study these problems and provide solutions to each. Firstly, we show that the matrices of the homography coefficients can be recast into two parts, each consistent with ranks of only one. This immediately establishes the prospect of realistically (that is, with as few as only three or four homographies) exploiting the redundancies of the homographies over two views. We also tackle the remaining issue: correlated coefficients. We compare our approach with the “gold standard”; that is, non-linear bundle adjustment (initialized from the ground truth estimate—the ideal initialization). The results confirm our theory and show one can implement rank-constrained projection and come close to the gold standard in effectiveness. Indeed, our algorithm (by itself), or our algorithm further refined by a bundle adjustment stage; may be a practical algorithm: providing generally better results than the “standard” DLT (direct linear transformation) algorithm, and even better than the bundle adjustment result with the DLT result as the starting point. Our unoptimized version has roughly the same cost as bundle adjustment and yet can generally produce close to the “gold standard” estimate (as illustrated by comparison with bundle adjustment initialized from the ground truth). Independent of the merits or otherwise of our algorithm, we have illuminated why the naive approach of direct rank-projection is relatively doomed to failure. Moreover, in revealing that there are further rank constraints, not previously known; we have added to the understanding of these issues, and this may pave the way for further improvements.     

基于SURF与光流法的增强现实跟踪注册

提出一种基于SURF与光流法相结合的增强现实局部跟踪注册方法。采用光流法对移动对象区域进行跟踪,利用SURF算法仿射、尺度不变性及运算速度快的优点对该区域进行特征提取与匹配,利用相邻帧之间特征点的匹配关系求得三维注册矩阵,在保持注册精确性的同时降低了系统运算时间。实验结果表明该方法达到了实时跟踪与准确注册的效果,并且在环境变化时保持了较好的鲁棒性。     

同形映射下的曲面场景的匹配算法的性能分析

针对移动机器人在曲面场景的匹配问题中,同形约束用于解决极约束产生的匹配模糊性问题和发现新的匹配点.实际上是平面块对曲面进行近似逼近的过程.逼近程度和逼近性能需要有指标进行定性和定量的衡量.故提出了两种性能评价指标:平均映射误差和平均映射匹配对.仿真实验结果的分析证明,场景深度变化或者场景距离摄像机的距离变化,对立体匹配算法性能本身不受影响,但映射和建立匹配关系时所需要的同形矩阵的数量不同.而且,随着特征点的稠密度提高,曲面场景的稳定性降低.可随着迭代过程的进行,算法本身结果还趋于稳定.     

Vision-based multi-UAV position estimation

This paper describes a method for vision-based unmanned aerial vehicle (UAV) motion estimation from multiple planar homographies. The paper also describes the determination of the relative displacement between different UAVs employing techniques for blob feature extraction and matching. It then presents and shows experimental results of the application of the proposed technique to multi-UAV detection of forest fires.     

Low‐altitude Terrestrial Spectroscopy from a Pushbroom Sensor

Hyperspectral cameras sample many different spectral bands at each pixel, enabling advanced detection and classification algorithms. However, their limited spatial resolution and the need to measure the camera motion to create hyperspectral images makes them unsuitable for nonsmooth moving platforms such as unmanned aerial vehicles (UAVs). We present a procedure to build hyperspectral images from line sensor data without camera motion information or extraneous sensors. Our approach relies on an accompanying conventional camera to exploit the homographies between images for mosaic construction. We provide experimental results from a low‐altitude UAV, achieving high‐resolution spectroscopy with our system.     

ReigSAC: fast discrimination of spurious keypoint correspondences on planar surfaces

Various methods were proposed to detect/match special interest points (keypoints) in images and some of them (e.g., SIFT and SURF) are among the most cited techniques in computer vision research. This paper describes an algorithm to discriminate between genuine and spurious keypoint correspondences on planar surfaces. We draw random samples of the set of correspondences, from which homographies are obtained and their principal eigenvectors extracted. Density estimation on that feature space determines the most likely true transform. Such homography feeds a cost function that gives the goodness of each keypoint correspondence. Being similar to the well-known RANSAC strategy, the key finding is that the main eigenvector of the most (genuine) homographies tends to represent a similar direction. Hence, density estimation in the eigenspace dramatically reduces the number of transforms actually evaluated to obtain reliable estimations. Our experiments were performed on hard image data sets, and pointed that the proposed approach yields effectiveness similar to the RANSAC strategy, at significantly lower computational burden, in terms of the proportion between the number of homographies generated and those that are actually evaluated.     

From lines to epipoles through planes in two views

This paper addresses the computation of the fundamental matrix between two views, when camera motion and 3D structure are unknown, but planar surfaces can be assumed. We use line features which are automatically matched in two steps. Firstly, with image based parameters, a set of matches are obtained to secondly compute homographies, which allows to reject wrong ones, and to grow good matches in a final stage. The inclusion of projective transformations gives much better results to match features with short computing overload. When two or more planes are observed, different homographies can be computed, segmenting simultaneously the corresponding planar surfaces. These can be used to obtain the fundamental matrix, which gives constraints for the whole scene. The results show that the global process is robust enough, turning out stable and useful to obtain matches and epipolar geometry from lines in man made environments.     

Dense mirroring surface recovery from 1D homographies and sparse correspondences

In this work, we recover the 3D shape of mirrors, sunglasses, and stainless steel implements. A computer monitor displays several images of parallel stripes, each image at a different angle. Reflections of these stripes in a mirroring surface are captured by the camera. For every image point, the direction of the displayed stripes and their reflections in the image are related by a 1D homography matrix, computed with a robust version of the statistically accurate heteroscedastic approach. By focusing on a sparse set of image points for which monitor-image correspondence is computed, the depth and the local shape may be estimated from these homographies. The depth estimation relies on statistically correct minimization and provides accurate, reliable results. Even for the image points where the depth estimation process is inherently unstable, we are able to characterize this instability and develop an algorithm to detect and correct it. After correcting the instability, dense surface recovery of mirroring objects is performed using constrained interpolation, which does not simply interpolate the surface depth values but also uses the locally computed 1D homographies to solve for the depth, the correspondence, and the local surface shape. The method was implemented and the shape of several objects was densely recovered at submillimeter accuracy.     

Observer‐based finite‐time output‐feedback controller for DC‐DC buck converters with unknown load variations

The output voltage regulation problem of a DC‐DC buck converter is investigated in this paper via an observer‐based finite‐time output‐feedback control approach. Considering the effects of unknown load variations and the case without current sensor, by using the technique of adding a power integrator and the idea of nonseparation principle, a finite‐time voltage regulation control algorithm via dynamic output feedback is designed. The main feature of the designed observer and controller does not need any load's information. Theoretically, it is proven that the output voltage can reach the desired voltage in a finite time under the proposed controller. The effectiveness of the proposed control method is illustrated by numerical simulations and experimental results.     

Plane-Based Calibration for Linear Cameras

Linear or 1D cameras are used in several areas such as industrial inspection and satellite imagery. Since 1D cameras consist of a linear sensor, a motion (usually perpendicular to the sensor orientation) is performed in order to acquire a full image. In this paper, we present a novel linear method to estimate the intrinsic and extrinsic parameters of a 1D camera using a planar object. As opposed to traditional calibration scheme based on 3D-2D correspondences of landmarks, our method uses homographies induced by the images of a planar object. The proposed algorithm is linear, simple and produces good results as shown by our experiments.     

An improved multiple‐state observer of Boolean control networks

This paper explores the issue of state estimation for Boolean control networks (BCNs), and a kind of improved multiple‐state observer is proposed. The improved multiple‐state observer can be described by means of a specific BCN that overcomes the difficulty of the existing multiple state observers where it is difficult to find a general expression for the observer gain matrix. Next, based on the states that can possibly generate the output and those that are observed by the designed observer in current time step, an adaptive algorithm that completes the design of the multiple‐state observer is provided to update the observer states, and which makes the state estimation of Boolean control networks feasible. Finally, an example is presented to illustrate the effectiveness of the obtained results.     

Robust and Efficient Feature Tracking for Indoor Navigation

Robust feature tracking is a requirement for many computer vision tasks such as indoor robot navigation. However, indoor scenes are characterized by poorly localizable features. As a result, indoor feature tracking without artificial markers is challenging and remains an attractive problem. We propose to solve this problem by constraining the locations of a large number of nondistinctive features by several planar homographies which are strategically computed using distinctive features. We experimentally show the need for multiple homographies and propose an illumination-invariant local-optimization scheme for motion refinement. The use of a large number of nondistinctive features within the constraints imposed by planar homographies allows us to gain robustness. Also, the lesser computation cost in estimating these nondistinctive features helps to maintain the efficiency of the proposed method. Our local-optimization scheme produces subpixel accurate feature motion. As a result, we are able to achieve robust and accurate feature tracking.     

Plane-based camera self-calibration by metric rectification of images

Plane-based self-calibration aims at the computation of camera intrinsic parameters from homographies relating multiple views of the same unknown planar scene. This paper proposes a straightforward geometric statement of plane-based self-calibration, through the concept of metric rectification of images. A set of constraints is derived from a decomposition of metric rectification in terms of intrinsic parameters and planar scene orientation. These constraints are then solved using an optimization framework based on the minimization of a geometrically motivated cost function. The link with previous approaches is demonstrated and our method appears to be theoretically equivalent but conceptually simpler. Moreover, a solution dealing with radial distortion is introduced. Experimentally, the method is compared with plane-based calibration and very satisfactory results are obtained. Markerless self-calibration is demonstrated using an intensity-based estimation of the inter-image homographies.     

Multiview constraints on homographies

The image motion of a planar surface between two camera views is captured by a homography (a 2D projective transformation). The homography depends on the intrinsic and extrinsic camera parameters, as well as on the 3D plane parameters. While camera parameters vary across different views, the plane geometry remains the same. Based on this fact, we derive linear subspace constraints on the relative homographies of multiple (⩾ 2) planes across multiple views. The paper has three main contributions: 1) We show that the collection of all relative homographies (homologies) of a pair of planes across multiple views, spans a 4-dimensional linear subspace. 2) We show how this constraint can be extended to the case of multiple planes across multiple views. 3) We show that, for some restricted cases of camera motion, linear subspace constraints apply also to the set of homographies of a single plane across multiple views. All the results derived are true for uncalibrated cameras. The possible utility of these multiview constraints for improving homography estimation and for detecting nonrigid motions are also discussed     

Constructive algorithm for dynamic observer error linearization via integrators: single output case

Dynamic observer error linearization which has been introduced recently is a new framework for observer design. Although this approach unifies several existing results on the problem and extends the class of systems that can be transformed into an observable linear system with an injection term of known signals, constructive algorithms to check the applicability are not available yet. In this paper, a constructive algorithm is proposed to solve the problem under some restrictions on the system structure and on the auxiliary dynamics introduced in the problem. The algorithm is constructive in the sense that the components of the transformation can be obtained step‐by‐step. Copyright © 2006 John Wiley & Sons, Ltd.