Guided Sampling via Weak Motion Models and Outlier Sample Generation for Epipolar Geometry Estimation

The problem of automatic robust estimation of the epipolar geometry in cases where the correspondences are contaminated with a high percentage of outliers is addressed. This situation often occurs when the images have undergone a significant deformation, either due to large rotation or wide baseline of the cameras. An accelerated algorithm for the identification of the false matches between the views is presented. The algorithm generates a set of weak motion models (WMMs). Each WMM roughly approximates the motion of correspondences from one image to the other. The algorithm represents the distribution of the median of the geometric distances of a correspondence to the WMMs as a mixture model of outlier correspondences and inlier correspondences. The algorithm generates a sample of outlier correspondences from the data. This sample is used to estimate the outlier rate and to estimate the outlier pdf. Using these two pdfs the probability that each correspondence is an inlier is estimated. These probabilities enable guided sampling. In the RANSAC process this guided sampling accelerates the search process. The resulting algorithm when tested on real images achieves a speedup of between one or two orders of magnitude. This work was supported partly by grant 01-99-08430 of the Israeli Space Agency through the Ministry of Science Culture and Sports of Israel.     

Adaptive robust estimation of affine parameters from block motion vectors

In this paper, we propose an affine parameter estimation algorithm from block motion vectors for extracting accurate motion information with the assumption that the undergoing motion can be characterized by an affine model. The motion may be caused either by a moving camera or a moving object. The proposed method first extracts motion vectors from a sequence of images by using size-variable block matching and then processes them by adaptive robust estimation to estimate affine parameters. Typically, a robust estimation filters out outliers (velocity vectors that do not fit into the model) by fitting velocity vectors to a predefined model. To filter out potential outliers, our adaptive robust estimation defines a continuous weight function based on a Sigmoid function. During the estimation process, we tune the Sigmoid function gradually to its hard-limit as the errors between the model and input data are decreased, so that we can effectively separate non-outliers from outliers with the help of the finally tuned hard-limit form of the weight function. Experimental results show that the suggested approach is very effective in estimating affine parameters reliably.     

Landsat-7 ETM+ radiometric normalization comparison for northern mapping applications

Relative radiometric normalization has long been performed to generate consistency among individual Landsat scenes for production of composites containing multiple scenes. Normalization methods have relied on matching identical and assumed invariant features in both images of an overlapping pair, or on invariant targets that are not necessarily the same features. Problems with overlap normalization methods include sensitivity to outliers in overlap data caused by atmospheric or land cover change between scenes, which can lead to radiometric error propagation across a mosaic caused by a normalized scene becoming a reference for the subsequent scene entered into the mosaic. Solutions to such problems include interactive outlier removal to generate a normalization function using a ‘no change’ data set and methods that are robust against outliers to automatically generate normalization functions with minimal user input. This paper compares two normalization methods that use a robust regression technique called Theil-Sen with an established overlap normalization method. The first method uses Theil-Sen regression to generate a normalization function between overlap regions, while the second uses Theil-Sen to normalize to coarse-resolution composite reflectance data from the SPOT VEGETATION (VGT) sensor. The results of the normalizations were evaluated in two ways: (1) using statistics generated between overlap regions; and (2) separately using coarse-resolution data as a reference. Both overlap normalization methods performed almost identically; however, Theil-Sen was faster and easier to implement than its traditional counterpart due to its insensitivity to outliers and capability for full automation. While overlap and coarse-resolution normalizations each outperformed the other when evaluated against its calibration set, error propagation caused by outliers in overlap samples was avoided in the normalization to coarse-resolution imagery. Advantages offered by normalization to coarse-resolution data using robust regression, including full automation, make this method particularly attractive for generation of large area mosaics containing 100 Landsat scenes or more.     

1-Point-RANSAC Structure from Motion for Vehicle-Mounted Cameras by Exploiting Non-holonomic Constraints

This paper presents a new method to estimate the relative motion of a vehicle from images of a single camera. The computational cost of the algorithm is limited only by the feature extraction and matching process, as the outlier removal and the motion estimation steps take less than a fraction of millisecond with a normal laptop computer. The biggest problem in visual motion estimation is data association; matched points contain many outliers that must be detected and removed for the motion to be accurately estimated. In the last few years, a very established method for removing outliers has been the “5-point RANSAC” algorithm which needs a minimum of 5 point correspondences to estimate the model hypotheses. Because of this, however, it can require up to several hundreds of iterations to find a set of points free of outliers. In this paper, we show that by exploiting the nonholonomic constraints of wheeled vehicles it is possible to use a restrictive motion model which allows us to parameterize the motion with only 1 point correspondence. Using a single feature correspondence for motion estimation is the lowest model parameterization possible and results in the two most efficient algorithms for removing outliers: 1-point RANSAC and histogram voting. To support our method we run many experiments on both synthetic and real data and compare the performance with a state-of-the-art approach. Finally, we show an application of our method to visual odometry by recovering a 3 Km trajectory in a cluttered urban environment and in real-time.     

Robust multi-view feature matching from multiple unordered views

This paper explores the problem of multi-view feature matching from an unordered set of widely separated views. A set of local invariant features is extracted independently from each view. First we propose a new view-ordering algorithm that organizes all the unordered views into clusters of related (i.e. the same scene) views by efficiently computing the view-similarity values of all view pairs by reasonably selecting part of extracted features to match. Second a robust two-view matching algorithm is developed to find initial matches, then detect the outliers and finally incrementally find more reliable feature matches under the epipolar constraint between two views from dense to sparse based on an assumption that changes of both motion and feature characteristics of one match are consistent with those of neighbors. Third we establish the reliable multi-view matches across related views by reconstructing missing matches in a neighboring triple of views and efficiently determining the states of matches between view pairs. Finally, the reliable multi-view matches thus obtained are used to automatically track all the views by using a self-calibration method. The proposed methods were tested on several sets of real images. Experimental results show that it is efficient and can track a large set of multi-view feature matches across multiple widely separated views.     

A two‐stage sampling for robust feature matching

For any visual feature‐based SLAM (simultaneous localization and mapping) solutions, to estimate the relative camera motion between two images, it is necessary to find “correct” correspondence between features extracted from those images. Given a set of feature correspondents, one can use a n‐point algorithm with robust estimation method, to produce the best estimate to the relative camera pose. The accuracy of a motion estimate is heavily dependent on the accuracy of the feature correspondence. Such a dependency is even more significant when features are extracted from the images of the scenes with drastic changes in viewpoints and illuminations and presence of occlusions. To make a feature matching robust to such challenging scenes, we propose a new feature matching method that incrementally chooses a five pairs of matched features for a full DoF (degree of freedom) camera motion estimation. In particular, at the first stage, we use our 2‐point algorithm to estimate a camera motion and, at the second stage, use this estimated motion to choose three more matched features. In addition, we use, instead of the epipolar constraint, a planar constraint for more accurate outlier rejection. With this set of five matching features, we estimate a full DoF camera motion with scale ambiguity. Through the experiments with three, real‐world data sets, our method demonstrates its effectiveness and robustness by successfully matching features (1) from the images of a night market where presence of frequent occlusions and varying illuminations, (2) from the images of a night market taken by a handheld camera and by the Google street view, and (3) from the images of a same location taken daytime and nighttime.     

采用逐层迭代方法的基础矩阵估计

为解决未知环境下运动序列中的基础矩阵估计问题,提出了一种逐层迭代优化的方法。该方法基于最优鲁棒估计方法,加入运动连续性以及多尺度对应的约束条件以减少虚假对应;然后,逐层将高层模型的数据内点添加到下层数据集,以更新数据集并同时估计单应性模型;最终,在底层全局优化并修正模型。实验表明,该方法的几何变换误差的均值不大于2.891821pixel,误差波动范围的方差不大于0.295172pixel,相对于传统方法,当运动序列中场景表面的深度层次较多,深度变化连续时,误差均值及波动方差均有一定程度的降低。     

分裂合并运动分割的多运动视觉里程计方法

目的 视觉里程计（visual odometry,VO）仅需要普通相机即可实现精度可观的自主定位,已经成为计算机视觉和机器人领域的研究热点,但是当前研究及应用大多基于场景为静态的假设,即场景中只有相机运动这一个运动模型,无法处理多个运动模型,因此本文提出一种基于分裂合并运动分割的多运动视觉里程计方法,获得场景中除相机运动外多个运动目标的运动状态。方法 基于传统的视觉里程计框架,引入多模型拟合的方法分割出动态场景中的多个运动模型,采用RANSAC（random sample consensus）方法估计出多个运动模型的运动参数实例;接着将相机运动信息以及各个运动目标的运动信息转换到统一的坐标系中,获得相机的视觉里程计结果,以及场景中各个运动目标对应各个时刻的位姿信息;最后采用局部窗口光束法平差直接对相机的姿态以及计算出来的相机相对于各个运动目标的姿态进行校正,利用相机运动模型的内点和各个时刻获得的相机相对于运动目标的运动参数,对多个运动模型的轨迹进行优化。结果 本文所构建的连续帧运动分割方法能够达到较好的分割结果,具有较好的鲁棒性,连续帧的分割精度均能达到近100%,充分保证后续估计各个运动模型参数的准确性。本文方法不仅能够有效估计出相机的位姿,还能估计出场景中存在的显著移动目标的位姿,在各个分段路径中相机自定位与移动目标的定位结果位置平均误差均小于6%。结论 本文方法能够同时分割出动态场景中的相机自身运动模型和不同运动的动态物体运动模型,进而同时估计出相机和各个动态物体的绝对运动轨迹,构建出多运动视觉里程计过程。     

基于有向模板和颜色加权SVD的图像匹配算法

为快速稳定地匹配视频序列,并考虑SVD算法的高效性,根据视频序列的特点,对SVD匹配算法进行改进,提出了一种适合视频序列的匹配算法。该算法使用Harris角点检测算子检测兴趣点,使用有向模板提取具有旋转不变性的特征,并通过引入颜色加权法改进SVD算法中的相似性度量函数。同时,又提出一种基于运动一致性约束的误配点剔除方法,首先拟合匹配点间的运动模型,然后自适应地调整参数将错误的匹配点剔除。该算法使用有向模板消除图像间旋转变换的影响,使用颜色特征降低兴趣点匹配时的不确定性,通过运动一致性约束降低误配点数量。实验结果表明,该算法在图像间存在旋转变换关系和不同的光照条件时都可以获得很好的匹配结果,特别是在图像间基线距离较大时仍能得到大量的匹配点并具有很高的正确匹配率,能很好满足实际需要。     

Robust Detection of Degenerate Configurations while Estimating the Fundamental Matrix

We present a new method for the detection of multiple solutions or degeneracy when estimating thefundamental matrix, with specific emphasis on robustness to data contamination (mismatches). The fundamental matrix encapsulates all the information on camera motion and internal parameters available from image feature correspondences between two views. It is often used as a first step in structure from motion algorithms. If the set of correspondences is degenerate, then this structure cannot be accurately recovered and many solutions explain the data equally well. It is essential that we are alerted to such eventualities. As current feature matchers are very prone to mismatching the degeneracy detection method must also be robust to outliers.In this paper a definition of degeneracy is given and all two-view nondegenerate and degenerate cases are catalogued in a logical way by introducing the language of varieties from algebraic geometry. It is then shown how each of the cases can be robustly determined from image correspondences via a scoring function we develop. These ideas define a methodology which allows the simultaneous detection of degeneracy and outliers. The method is called PLUNDER-DL and is a generalization of the robust estimator RANSAC.The method is evaluated on many differing pairs of real images. In particular it is demonstrated that proper modeling of degeneracy in the presence of outliers enables the detection of mismatches which would otherwise be missed. All processing including point matching, degeneracy detection, and outlier detection is automatic.     

Automatic reconstruction of 3D human motion pose from uncalibrated monocular video sequences based on markerless human motion tracking

We present a method to reconstruct human motion pose from uncalibrated monocular video sequences based on the morphing appearance model matching. The human pose estimation is made by integrated human joint tracking with pose reconstruction in depth-first order. Firstly, the Euler angles of joint are estimated by inverse kinematics based on human skeleton constrain. Then, the coordinates of pixels in the body segments in the scene are determined by forward kinematics, by projecting these pixels in the scene onto the image plane under the assumption of perspective projection to obtain the region of morphing appearance model in the image. Finally, the human motion pose can be reconstructed by histogram matching. The experimental results show that this method can obtain favorable reconstruction results on a number of complex human motion sequences.     

Performance evaluation of 1‐point‐RANSAC visual odometry

Monocular visual odometry is the process of computing the egomotion of a vehicle purely from images of a single camera. This process involves extracting salient points from consecutive image pairs, matching them, and computing the motion using standard algorithms. This paper analyzes one of the most important steps toward accurate motion computation, which is outlier removal. The random sample consensus (RANSAC) has been established as the standard method for model estimation in the presence of outliers. RANSAC is an iterative method, and the number of iterations necessary to find a correct solution is exponential in the minimum number of data points needed to estimate the model. It is therefore of utmost importance to find the minimal parameterization of the model to estimate. For unconstrained motion [six degrees of freedom (DoF)] of a calibrated camera, this would be five correspondences. In the case of planar motion, the motion model complexity is reduced (three DoF) and can be parameterized with two points. In this paper we show that when the camera is installed on a nonholonomic wheeled vehicle, the model complexity reduces to two DoF and therefore the motion can be parameterized with a single‐point correspondence. Using a single‐feature correspondence for motion estimation is the lowest model parameterization possible and results in the most efficient algorithm for removing outliers, which we call 1‐point RANSAC. To support our method, we run many experiments on both synthetic and real data and compare the performance with state‐of‐the‐art approaches and with different vehicles, both indoors and outdoors. © 2011 Wiley Periodicals, Inc.     

Accurate and fast 3D head pose estimation with noisy RGBD images

Head pose estimation plays an essential role in many high-level face analysis tasks. However, accurate and robust pose estimation with existing approaches remains challenging. In this paper, we propose a novel method for accurate three-dimensional (3D) head pose estimation with noisy depth maps and high-resolution color images that are typically produced by popular RGBD cameras such as the Microsoft Kinect. Our method combines the advantages of the high-resolution RGB image with the 3D information of the depth image. For better accuracy and robustness, features are first detected using only the color image, and then the 3D feature points used for matching are obtained by combining depth information. The outliers are then filtered with depth information using rules proposed for depth consistency, normal consistency, and re-projection consistency, which effectively eliminate the influence of depth noise. The pose parameters are then iteratively optimized using the Extended LM (Levenberg-Marquardt) method. Finally, a Kalman filter is used to smooth the parameters. To evaluate our method, we built a database of more than 10K RGBD images with ground-truth poses recorded using motion capture. Both qualitative and quantitative evaluations show that our method produces notably smaller errors than previous methods.     

Robust online appearance models for visual tracking

We propose a framework for learning robust, adaptive, appearance models to be used for motion-based tracking of natural objects. The model adapts to slowly changing appearance, and it maintains a natural measure of the stability of the observed image structure during tracking. By identifying stable properties of appearance, we can weight them more heavily for motion estimation, while less stable properties can be proportionately downweighted. The appearance model involves a mixture of stable image structure, learned over long time courses, along with two-frame motion information and an outlier process. An online EM-algorithm is used to adapt the appearance model parameters over time. An implementation of this approach is developed for an appearance model based on the filter responses from a steerable pyramid. This model is used in a motion-based tracking algorithm to provide robustness in the face of image outliers, such as those caused by occlusions, while adapting to natural changes in appearance such as those due to facial expressions or variations in 3D pose.     

基于引力模型的三维运动点集的非精确匹配

本文提出了一种基于引力模型(attractive model)的非精确匹配算法,应用于三维空间运动点集的对应点匹配问题.根据引力模型,我们将匹配和运动估计问题转化为一个代价函数的全局优化问题,实现了无对应点的运动估计和总体匹配.这种算法是一个鲁棒(robust)估计和匹配方法,可以处理包含非匹配点对的三维运动点集.大量计算机模拟实验结果充分证明了算法的鲁棒性和有效性.     

A robust scheme of model parameters estimation based on the particle swarm method in the image matching problem

A new iterative method for robust estimation of image transform parameters based on particle swarm optimization is proposed. The main distinction of the method from the RANSAC method of random search which is frequently applied to solving problems of robust parameters estimation in computer vision problems, consists in the fact that at each iteration the test samples are generated with taking into account the information about model quality, constructed based on samples at all previous iterations, rather than randomly. The rules for refinement of samples are motivated by the behavior of swarm (schooling) living creatures. The efficiency of the new SwarmSAC algorithm is illustrated by an example of stereo matching of two images when matching errors (outliers) are present. The results of comparison of the algorithm with the RANSAC method demonstrate the advantage of the new algorithm in solving the image matching problem. The new method is generic and can be applied to various problems of robust parameters estimation of parameters and filtering of outliers.     

Accurate keyframe selection and keypoint tracking for robust visual odometry

This paper presents a novel stereo visual odometry (VO) framework based on structure from motion, where a robust keypoint tracking and matching is combined with an effective keyframe selection strategy. In order to track and find correct feature correspondences a robust loop chain matching scheme on two consecutive stereo pairs is introduced. Keyframe selection is based on the proportion of features with high temporal disparity. This criterion relies on the observation that the error in the pose estimation propagates from the uncertainty of 3D points—higher for distant points, that have low 2D motion. Comparative results based on three VO datasets show that the proposed solution is remarkably effective and robust even for very long path lengths.     

基于动态物体跟踪的语义SLAM

针对传统视觉SLAM在动态场景下容易出现特征匹配错误从而导致定位精度下降的问题,提出了一种基于动态物体跟踪的语义SLAM算法。基于经典的视觉SLAM框架,提取动态物体进行帧间跟踪,并利用动态物体的位姿信息来辅助相机自身的定位。首先,算法在数据预处理中使用YOLACT、RAFT以及SC-Depth网络,分别提取图像中的语义掩膜、光流向量以及像素深度值。其次,视觉前端模块根据所提信息,通过语义分割掩膜、运动一致性检验以及遮挡点检验算法计算概率图以平滑区分场景中的动态特征与静态特征。然后,后端中的捆集调整模块融合了物体运动的多特征约束以提高算法在动态场景中的位姿估计性能。最后,在KITTI和OMD数据集的动态场景中进行对比验证。实验表明,所提算法能够准确地跟踪动态物体,在室内外动态场景中具备鲁棒、良好的定位性能。     

The background primal sketch: An approach for tracking moving objects

In this paper we present an algorithm that integrates spatial and temporal information for the tracking of moving nonrigid objects. In addition, we obtain outlines of the moving objects.Three basic ingredients are employed in the proposed algorithm, namely, the background primal sketch, the threshold, and outlier maps. The background primal sketch is an edge map of the background without moving objects. If the background primal sketch is known, then edges of moving objects can be determined by comparing the edge map of the input image with the background primal sketch. A moving edge point is modeled as an outlier, that is, a pixel with an edge value differing from the background edge value in the background primal sketch by an amount larger than the threshold in the threshold map at the same physical location. The map that contains all the outliers is called the outlier map. In this paper we present techniques based on robust statistics for determining the background primal sketch, the threshold, and outlier maps.In an ideal situation the outlier map would contain the complete outlines of the moving objects. In practice, the outliers do not form closed contours. The final step of the algorithm employs an edge-guided morphological approach to generate closed outlines of the moving objects. The proposed approach has been tested on sequences of moving human blood cells (neutrophil) as well as of human body motion with encouraging results.     

Bias in robust estimation caused by discontinuities and multiplestructures

When fitting models to data containing multiple structures, such as when fitting surface patches to data taken from a neighborhood that includes a range discontinuity, robust estimators must tolerate both gross outliers and pseudo outliers. Pseudo outliers are outliers to the structure of interest, but inliers to a different structure. They differ from gross outliers because of their coherence. Such data occurs frequently in computer vision problems, including motion estimation, model fitting, and range data analysis. The focus in this paper is the problem of fitting surfaces near discontinuities in range data. To characterize the performance of least median of the squares, least trimmed squares, M-estimators, Hough transforms, RANSAC, and MINPRAN on this type of data, the “pseudo outlier bias” metric is developed using techniques from the robust statistics literature, and it is used to study the error in robust fits caused by distributions modeling various types of discontinuities. The results show each robust estimator to be biased at small, but substantial, discontinuities. They also show the circumstances under which different estimators are most effective. Most importantly, the results imply present estimators should be used with care, and new estimators should be developed