Performance evaluation of 1‐point‐RANSAC visual odometry

Monocular visual odometry is the process of computing the egomotion of a vehicle purely from images of a single camera. This process involves extracting salient points from consecutive image pairs, matching them, and computing the motion using standard algorithms. This paper analyzes one of the most important steps toward accurate motion computation, which is outlier removal. The random sample consensus (RANSAC) has been established as the standard method for model estimation in the presence of outliers. RANSAC is an iterative method, and the number of iterations necessary to find a correct solution is exponential in the minimum number of data points needed to estimate the model. It is therefore of utmost importance to find the minimal parameterization of the model to estimate. For unconstrained motion [six degrees of freedom (DoF)] of a calibrated camera, this would be five correspondences. In the case of planar motion, the motion model complexity is reduced (three DoF) and can be parameterized with two points. In this paper we show that when the camera is installed on a nonholonomic wheeled vehicle, the model complexity reduces to two DoF and therefore the motion can be parameterized with a single‐point correspondence. Using a single‐feature correspondence for motion estimation is the lowest model parameterization possible and results in the most efficient algorithm for removing outliers, which we call 1‐point RANSAC. To support our method, we run many experiments on both synthetic and real data and compare the performance with state‐of‐the‐art approaches and with different vehicles, both indoors and outdoors. © 2011 Wiley Periodicals, Inc.     

A two‐stage sampling for robust feature matching

For any visual feature‐based SLAM (simultaneous localization and mapping) solutions, to estimate the relative camera motion between two images, it is necessary to find “correct” correspondence between features extracted from those images. Given a set of feature correspondents, one can use a n‐point algorithm with robust estimation method, to produce the best estimate to the relative camera pose. The accuracy of a motion estimate is heavily dependent on the accuracy of the feature correspondence. Such a dependency is even more significant when features are extracted from the images of the scenes with drastic changes in viewpoints and illuminations and presence of occlusions. To make a feature matching robust to such challenging scenes, we propose a new feature matching method that incrementally chooses a five pairs of matched features for a full DoF (degree of freedom) camera motion estimation. In particular, at the first stage, we use our 2‐point algorithm to estimate a camera motion and, at the second stage, use this estimated motion to choose three more matched features. In addition, we use, instead of the epipolar constraint, a planar constraint for more accurate outlier rejection. With this set of five matching features, we estimate a full DoF camera motion with scale ambiguity. Through the experiments with three, real‐world data sets, our method demonstrates its effectiveness and robustness by successfully matching features (1) from the images of a night market where presence of frequent occlusions and varying illuminations, (2) from the images of a night market taken by a handheld camera and by the Google street view, and (3) from the images of a same location taken daytime and nighttime.     

Finding trajectories of feature points in a monocular image sequence

Identifying the same physical point in more than one image, the correspondence problem, is vital in motion analysis. Most research for establishing correspondence uses only two frames of a sequence to solve this problem. By using a sequence of frames, it is possible to exploit the fact that due to inertia the motion of an object cannot change instantaneously. By using smoothness of motion, it is possible to solve the correspondence problem for arbitrary motion of several nonrigid objects in a scene. We formulate the correspondence problem as an optimization problem and propose an iterative algorithm to find trajectories of points in a monocular image sequence. A modified form of this algorithm is useful in case of occlusion also. We demonstrate the efficacy of this approach considering synthetic, laboratory, and real scenes.     

Multi-sensor based real-time 6-DoF pose tracking for wearable augmented reality

Wearable augmented reality (WAR) combines a live view of a real scene with computer-generated graphic on resource-limited platforms. One of the crucial technologies for WAR is a real-time 6-DoF pose tracking, facilitating registration of virtual components within in a real scene. Generally, artificial markers are typically applied to provide pose tracking for WAR applications. However, these marker-based methods suffer from marker occlusions or large viewpoint changes. Thus, a multi-sensor based tracking approach is applied in this paper, and it can perform real-time 6-DoF pose tracking with real-time scale estimation for WAR on a consumer smartphone. By combining a wide-angle monocular camera and an inertial sensor, a more robust 6-DoF motion tracking is demonstrated with the mutual compensations of the heterogeneous sensors. Moreover, with the help of the depth sensor, the scale initialization of the monocular tracking is addressed, where the initial scale is propagated within the subsequent sensor-fusion process, alleviating the scale drift in traditional monocular tracking approaches. In addition, a sliding-window based Kalman filter framework is used to provide a low jitter pose tracking for WAR. Finally, experiments are carried out to demonstrate the feasibility and robustness of the proposed tracking method for WAR applications.     

人体运动分析的实例学习方法

目的 面向实时、准确、鲁棒的人体运动分析应用需求,从运动分析的特征提取和运动建模问题出发,本文人体运动分析的实例学习方法。方法 在构建人体姿态实例库基础上,首先,采用运动检测方法得到视频每帧的人体轮廓;其次,基于形状上下文轮廓匹配方法,从实例库中检索得到每帧视频的候选姿态集;最后,通过统计建模和转移概率建模实现人体运动分析。结果 对步行、跑步、跳跃等测试视频进行实验,基于轮廓的形状上下文特征表示和匹配方法具有良好的表达能力;本文方法运动分析结果,关节夹角平均误差在5°左右,与其他算法相比,有效提高了运动分析的精度。结论 本文人体运动分析的实例学习方法,能有效分析单目视频中的人体运动,并克服了映射的深度歧义,对运动的视角变化鲁棒,具有良好的计算效率和精度。     

Feature correspondence and motion recovery in vehicle planar navigation

This paper describes a strategy to feature point correspondence and motion recovery in vehicle navigation. A transformation of the image plane is proposed that keeps the motion of the vehicle on a plane parallel to the transformed image plane. This permits to define linear tracking filters to estimate the real-world positions of the features, and allows us to select the matches that accomplish the rigidity of the scene by a Hough transform. Candidate correspondences are selected by similarity, taking into account the smoothness of motion. Further processing brings out the final matching. The methods have been tested in a real application.     

Quasi-plus sampling edge correction for spatial point patterns

A widely applicable edge correction method for estimating summary statistics of a spatial point pattern is proposed. We reconstruct point patterns in a larger region containing the sampling window by matching sampled and simulated kth nearest neighbour distance distributions of the given pattern and then apply plus sampling. Simulation studies show that this approach, called quasi-plus sampling, gives estimates with smaller root mean squared errors than estimates obtained by using other popular edge corrections. We apply the proposed approach to real data and yield an estimate of a summary statistic that is more plausible than that obtained by a popular edge correction.     

1-Point-RANSAC Structure from Motion for Vehicle-Mounted Cameras by Exploiting Non-holonomic Constraints

This paper presents a new method to estimate the relative motion of a vehicle from images of a single camera. The computational cost of the algorithm is limited only by the feature extraction and matching process, as the outlier removal and the motion estimation steps take less than a fraction of millisecond with a normal laptop computer. The biggest problem in visual motion estimation is data association; matched points contain many outliers that must be detected and removed for the motion to be accurately estimated. In the last few years, a very established method for removing outliers has been the “5-point RANSAC” algorithm which needs a minimum of 5 point correspondences to estimate the model hypotheses. Because of this, however, it can require up to several hundreds of iterations to find a set of points free of outliers. In this paper, we show that by exploiting the nonholonomic constraints of wheeled vehicles it is possible to use a restrictive motion model which allows us to parameterize the motion with only 1 point correspondence. Using a single feature correspondence for motion estimation is the lowest model parameterization possible and results in the two most efficient algorithms for removing outliers: 1-point RANSAC and histogram voting. To support our method we run many experiments on both synthetic and real data and compare the performance with a state-of-the-art approach. Finally, we show an application of our method to visual odometry by recovering a 3 Km trajectory in a cluttered urban environment and in real-time.     

Real time egomotion of a nonholonomic vehicle using LIDAR measurements

This paper presents a technique to estimate in real time the egomotion of a vehicle based solely on laser range data. This technique calculates the discrepancy between closely spaced two‐dimensional laser scans due to the vehicle motion using scan matching techniques. The result of the scan alignment is converted into a nonlinear motion measurement and fed into a nonholonomic extended Kalman filter model. This model better approximates the real motion of the vehicle when compared to more simplistic models, thus improving performance and immunity to outliers. The motion estimate is intended to be used for egomotion compensation in a target‐tracking algorithm for situation awareness applications. In this paper, several recent scan matching algorithms were evaluated for their accuracy and computational speed: metric‐based iterative closest point (MbICP), point‐to‐line ICP (PIICP), and polar scan matching. The proposed approach is performed in real time and provides an accurate estimate of the current robot motion. The MbICP algorithm proved to be the most advantageous scan matching algorithm, but it is still comparable to PlICP. The motion estimation algorithm is validated through experimental testing in real world conditions.     

基于单目视觉的车辆下边沿估计和逆透视变换的车距测量

前方车辆测距在自动驾驶汽车技术领域中起着至关重要的作用。针对目前基于单目视觉的车辆测距技术忽略了车辆与地面相接的下边沿问题,提出一种基于车辆下边沿估计和逆透视变换的单目视觉测距方法,实现了对前方车辆进行横向和纵向的高精度车距测量。该方法首先通过对车辆关键点估计和几何关系模型完成对车辆下边沿的估计,然后从中计算测距关键点,再利用基于点的逆透视变换测距模型进行距离计算。实验结果表明,与其他基于单目视觉的车辆测距方法相比,该方法提高了测距的精度和稳定性。     

Model-based object tracking in monocular image sequences of road traffic scenes

Moving vehicles are detected and tracked automatically in monocular image sequences from road traffic scenes recorded by a stationary camera. In order to exploit the a priori knowledge about shape and motion of vehicles in traffic scenes, a parameterized vehicle model is used for an intraframe matching process and a recursive estimator based on a motion model is used for motion estimation. An interpretation cycle supports the intraframe matching process with a state MAP-update step. Initial model hypotheses are generated using an image segmentation component which clusters coherently moving image features into candidate representations of images of a moving vehicle. The inclusion of an illumination model allows taking shadow edges of the vehicle into account during the matching process. Only such an elaborate combination of various techniques has enabled us to track vehicles under complex illumination conditions and over long (over 400 frames) monocular image sequences. Results on various real-world road traffic scenes are presented and open problems as well as future work are outlined.     

Cooperative Stereo–Motion: Matching and Reconstruction

One of the most interesting goals of computer vision is the 3D structure recovery of scenes. Traditionally, two cues are used: structure from motion and structure from stereo, two subfields with complementary sets of assumptions and techniques. This paper introduces a new general framework of cooperation between stereo and motion. This framework combines the advantages of both cues: (i) easy correspondence from motion and (ii) accurate 3D reconstruction from stereo. First, we show how the stereo matching can be recovered from motion correspondences using only geometric constraints. Second, we propose a method of 3D reconstruction of both binocular and monocular features using all stereo pairs in the case of a calibrated stereo rig. Third, we perform an analysis of the performance of the proposed framework as well as a comparison with an affine method. Experiments involving real and synthetic stereo pairs indicate that rich and reliable information can be derived from the proposed framework. They also indicate that robust 3D reconstruction can be obtained even with short image sequences.     

Scale-Free Registrations in 3D: 7 Degrees of Freedom with Fourier Mellin SOFT Transforms

Fourier Mellin SOFT (FMS) as a novel method for global registration of 3D data is presented. It determines the seven degrees of freedom (7-DoF) transformation, i.e., the 6-DoF rigid motion parameters plus 1-DoF scale, between two scans, i.e., two noisy, only partially overlapping views on objects or scenes. It is based on a sequence of the 3D Fourier transform, the Mellin transform and the SO(3) Fourier transform. This combination represents a non-trivial complete 3D extension of the well known Fourier-Mellin registration for 2D images. It is accordingly based on decoupling rotation and scale from translation. First, rotation—which is the main challenge for the extension to 3D data - is tackled with a SO(3) Fourier Transform (SOFT) based on Spherical Harmonics. In a second step, scale is determined via a 3D Mellin transform. Finally, translation is calculated by Phase-Matching. Experiments are presented with simulated data sets for ground truth comparisons and with real world data including object recognition and localization in Magnetic Resonance Tomography (MRT) data, registration of 2.5D RGBD scans from a Microsoft Kinect with a scale-free 3D model generated by Multi-View Vision, and 3D mapping by registration of a sequence of consecutive scans from a low-cost actuated Laser Range Finder. The results show that the method is fast and that it can robustly handle partial overlap, interfering structures, and noise. It is also shown that the method is a very interesting option for 6-DoF registration, i.e., when scale is known.     

A computer vision system for the detection and classification of vehicles at urban road intersections

The paper presents a real-time vision system to compute traffic parameters by analyzing monocular image sequences coming from pole-mounted video cameras at urban crossroads. The system uses a combination of segmentation and motion information to localize and track moving objects on the road plane, utilizing a robust background updating, and a feature-based tracking method. It is able to describe the path of each detected vehicle, to estimate its speed and to classify it into seven categories. The classification task relies on a model-based matching technique refined by a feature-based one for distinguishing between classes having similar models, like bicycles and motorcycles. The system is flexible with respect to the intersection geometry and the camera position. Experimental results demonstrate robust, real-time vehicle detection, tracking and classification over several hours of videos taken under different illumination conditions. The system is presently under trial in Trento, a 100,000-people town in northern Italy.     

Integrated 3-D analysis and analysis-guided synthesis of flightimage sequences

This paper is concerned with three-dimensional (3D) analysis, and analysis-guided syntheses, of images showing 3-D motion of an observer relative to a scene. There are two objectives of the paper. First, it presents an approach to recovering 3D motion and structure parameters from multiple cues present in a monocular image sequence, such as point features, optical flow, regions, lines, texture gradient, and vanishing line. Second, it introduces the notion that the cues that contribute the most to 3-D interpretation are also the ones that would yield the most realistic synthesis, thus suggesting an approach to analysis guided 3-D representation. For concreteness, the paper focuses on flight image sequences of a planar, textured surface. The integration of information in these diverse cues is carried out using optimization. For reliable estimation, a sequential batch method is used to compute motion and structure. Synthesis is done by using (i) image attributes extracted from the image sequence, and (ii) simple, artificial image attributes which are not present in the original images. For display, real and/or artificial attributes are shown as a monocular or a binocular sequence. Performance evaluation is done through experiments with one synthetic sequence, and two real image sequences digitized from a commercially available video tape and a laserdisc. The attribute based representation of these sequences compressed their sizes by 502 and 367. The visualization sequence appears very similar to the original sequence in informal, monocular as well as stereo viewing on a workstation monitor     

基于颜色和几何特征的图像特征点匹配算法

提出一种基于颜色和几何特征的图像特征点匹配算法。首先提取两幅图像特征点集邻域色调的局部累加直方图,然后结合图像特征点的几何特征构造亲近矩阵,再对亲近矩阵进行奇异值分解(SVD),利用分解的结果构造出一个反应特征点之间匹配程度的关系矩阵,最后根据关系矩阵实现两幅图像的特征点匹配。实验结果显示,这种图像特征点匹配算法对真实图像的平面旋转和立体旋转都具有较高的匹配精确度。     

Quantitative planar region detection

This paper presents a means of segmenting planar regions from two views of a scene using point correspondences. The initial selection of groups of coplanar points is performed on the basis of conservation of two five point projective invariants (groups for which this invariant is conserved are assumed to be coplanar). The five point correspondences are used to estimate a projectivity which is used to predict the change in position of other points assuming they lie on the same plane as the original four. The variance in any points new position is then used to define a distance threshold between actual and predicted position which is used as a coplanarity test to find extended planar regions. If two distinct planar regions can be found then a novel motion direction estimator suggests itself. The projection of the line of intersection of two planes in an image may also be recovered. An analytical error model is derived which relates image uncertainty in a corner's position to genuine perpendicular height of a point above a given plane in the world. The model may be used for example to predict the performance of given stereo ground plane prediction system or a monocular drivable region detection system on and AGV. The model may also be used in reverse to determine the camera resolution required if a vehicle in motion is to resolve obstacles of a given height a given distance from it.     

基于引力模型的三维运动点集的非精确匹配

本文提出了一种基于引力模型(attractive model)的非精确匹配算法,应用于三维空间运动点集的对应点匹配问题.根据引力模型,我们将匹配和运动估计问题转化为一个代价函数的全局优化问题,实现了无对应点的运动估计和总体匹配.这种算法是一个鲁棒(robust)估计和匹配方法,可以处理包含非匹配点对的三维运动点集.大量计算机模拟实验结果充分证明了算法的鲁棒性和有效性.