Bayesian modeling of uncertainty in low-level vision

The need for error modeling, multisensor fusion, and robust algorithms is becoming increasingly recognized in computer vision. Bayesian modeling is a powerful, practical, and general framework for meeting these requirements. This article develops a Bayesian model for describing and manipulating the dense fields, such as depth maps, associated with low-level computer vision. Our model consists of three components: a prior model, a sensor model, and a posterior model. The prior model captures a priori information about the structure of the field. We construct this model using the smoothness constraints from regularization to define a Markov Random Field. The sensor model describes the behavior and noise characteristics of our measurement system. We develop a number of sensor models for both sparse and dense measurements. The posterior model combines the information from the prior and sensor models using Bayes' rule. We show how to compute optimal estimates from the posterior model and also how to compute the uncertainty (variance) in these estimates. To demonstrate the utility of our Bayesian framework, we present three examples of its application to real vision problems. The first application is the on-line extraction of depth from motion. Using a two-dimensional generalization of the Kalman filter, we develop an incremental algorithm that provides a dense on-line estimate of depth whose accuracy improves over time. In the second application, we use a Bayesian model to determine observer motion from sparse depth (range) measurements. In the third application, we use the Bayesian interpretation of regularization to choose the optimal smoothing parameter for interpolation. The uncertainty modeling techniques that we develop, and the utility of these techniques in various applications, support our claim that Bayesian modeling is a powerful and practical framework for low-level vision.     

An adaptive observer framework for accurate feature depth estimation using an uncalibrated monocular camera

This paper presents a novel solution to the problem of depth estimation using a monocular camera undergoing known motion. Such problems arise in machine vision where the position of an object moving in three-dimensional space has to be identified by tracking motion of its projected feature on the two-dimensional image plane. The camera is assumed to be uncalibrated, and an adaptive observer yielding asymptotic estimates of focal length and feature depth is developed that precludes prior knowledge of scene geometry and is simpler than alternative designs. Experimental results using real camera imagery are obtained with the current scheme as well as the extended Kalman filter, and performance of the proposed observer is shown to be better than the extended Kalman filter-based framework.     

A Kalman filter approach to direct depth estimation incorporatingsurface structure

The problem of depth-from-motion using a monocular image sequence is considered. A pixel-based model is developed for direct depth estimation within a Kalman filtering framework. A method is proposed for incorporating local surface structure into the Kalman filter. Experimental results are provided to illustrate the effect of structural information on depth estimation     

An Approach Toward Visual Autonomous Ship Board Landing of a VTOL UAV

We present the design and implementation of a vision based autonomous landing algorithm using a downward looking camera. To demonstrate the efficacy of our algorithms we emulate the dynamics of the ship-deck, for various sea states and different ships using a six degrees of freedom motion platform. We then present the design and implementation of our robust computer vision system to measure the pose of the shipdeck with respect to the vehicle. A Kalman filter is used in conjunction with our vision system to ensure the robustness of the estimates. We demonstrate the accuracy and robustness of our system to occlusions, variation in intensity, etc. using our testbed.     

The Motor Extended Kalman Filter: A Geometric Approach for Rigid Motion Estimation

In this paper the motor algebra for linearizing the 3D Euclidean motion of lines is used as the oretical basis for the development of a novel extended Kalman filter called the motor extended Kalman filter (MEKF). Due to its nature the MEKF can be used as online approach as opposed to batch SVD methods. The MEKF does not encounter singularities when computing the Kalman gain and it can estimate simultaneously the translation and rotation transformations. Many algorithms in the literature compute the translation and rotation transformations separately. The experimental part demonstrates that the motor extended Kalman filter is an useful approach for estimation of dynamic motion problems. We compare the MEKF with an analytical method using simulated data. We present also an application using real images of a visual guided robot manipulator; the aim of this experiment is to demonstrate how we can use the online MEKF algorithm. After the system has been calibrated, the MEKF estimates accurately the relative position of the end-effector and a 3D reference line. We believe that future vision systems being reliably calibrated will certainly make great use of the MEKF algorithm.     

基于立体视觉的非合作航天器相对导航

非合作航天器其没有可交互的传感器,传统的相对导航方法难以满足精度要求。根据非合作航天器相对导航特点,提出一种基于立体视觉的空间非合作目标相对导航算法。方法使用安装在追踪航天器上的立体视觉相机作为测量传感器,实现目标在追踪航天器体坐标系下相对位置的测量。在惯性坐标系下建立航天器相对运动方程,并离散化作为系统的状态方程,利用立体视觉的测量信息作为量测值,在此基础上设计基于卡尔曼滤波(Kalman filter)的两步滤波相对导航算法,实现航天器间相对导航状态的实时估计,仿真结果显示算法实时性、有效性,导航方法收敛速度较快,精度高,满足相对导航要求。     

Recursive estimation of 3D motion and surface structure from local affine flow parameters

A recursive structure from motion algorithm based on optical flow measurements taken from an image sequence is described. It provides estimates of surface normal in addition to 3D motion and depth. The measurements are affine motion parameters which approximate the local flow fields associated with near-planar surface patches in the scene. These are integrated over time to give estimates of the 3D parameters using an extended Kalman filter. This also estimates the camera focal length and, so, the 3D estimates are metric. The use of parametric measurements means that the algorithm is computationally less demanding than previous optical flow approaches and the recursive filter builds in a degree of noise robustness. Results of experiments on synthetic and real image sequences demonstrate that the algorithm performs well.     

基于扩散卡尔曼滤波算法的目标跟踪估计

针对节点网络上的目标跟踪,提出一种基于扩散Kalman滤波算法的分布式跟踪估计.假设该节点网络系统按照线性状态空间模型演进,网络中的每个节点获取与未观察到的状态线性相关的测量值;对于每个测量值和每个节点,采用来自邻近区域的数据计算出一个局部状态估计值;采用一个基于扩散矩阵和连接矩阵的扩散步骤,将前面计算得到的邻域估计值...     

适用于户外增强现实系统的混合跟踪定位算法

单一传感器无法解决户外增强现实系统中的跟踪定位问题.为了提高视觉跟踪定位算法的精度和鲁棒性,提出一种基于惯性跟踪器与视觉测量相结合的混合跟踪定位算法.该算法在扩展卡尔曼滤波框架下,通过融合来自视觉与惯性传感器的信息进行摄像机运动轨迹估计,并利用视觉测量信息对惯性传感器的零点偏差进行实时校正;同时采用SCAAT方法解决惯性传感器与视觉测量间的时间采样不同步问题.实验结果表明,该算法能够有效地提高运动估计的精度和稳定性.     

Less restrictive camera odometry estimation from monocular camera

This paper addresses the problem of estimating a camera motion from a non-calibrated monocular camera. Compared to existing methods that rely on restrictive assumptions, we propose a method which can estimate camera motion with much less restrictions by adopting new example-based techniques compensating the lack of information. Specifically, we estimate the focal length of the camera by referring to visually similar training images with which focal lengths are associated. For one step camera estimation, we refer to stationary points (landmark points) whose depths are estimated based on RGB-D candidates. In addition to landmark points, moving objects can be also used as an information source to estimate the camera motion. Therefore, our method simultaneously estimates the camera motion for a video, and the 3D trajectories of objects in this video by using Reversible Jump Markov Chain Monte Carlo (RJ-MCMC) particle filtering. Our method is evaluated on challenging datasets demonstrating its effectiveness and efficiency.     

A closed-form expression for the uncertainty in odometry position estimate of an autonomous vehicle

Using internal and external sensors to provide position estimates in a two-dimensional space is necessary to solve the localization and navigation problems for a robot or an autonomous vehicle (AV). Usually, a unique source of position information is not enough, so researchers try to fuse data from different sensors using several methods, as, for example, Kalman filtering. Those methods need an estimation of the uncertainty in the position estimates obtained from the sensory system. This uncertainty is expressed by a covariance matrix, which is usually obtained from experimental data, assuming, by the nature of this matrix, general and unconstrained motion. We propose in this paper a closed-form expression for the uncertainty in the odometry position estimate of a mobile vehicle, using a covariance matrix whose form is derived from the cinematic model. We then particularize for a nonholonomic Ackerman driving-type AV. Its cinematic model relates the two measures being obtained for internal sensors: the velocity, translated into the instantaneous displacement; and the instantaneous steering angle. The proposed method is validated experimentally, and compared against Kalman filtering.     

Image motion estimation from motion smear-a new computational model

Motion smear is an important visual cue for motion perception by the human vision system (HVS). However, in image analysis research, exploiting motion smear has been largely ignored. Rather, motion smear is usually considered as a degradation of images that needs to be removed. In this paper, the authors establish a computational model that estimates image motion from motion smear information-“motion from smear”. In many real situations, the shutter of the sensing camera must be kept open long enough to produce images of adequate signal-to-noise ratio (SNR), resulting in significant motion smear in images. The authors present a new motion blur model and an algorithm that enables unique estimation of image motion. A prototype sensor system that exploits the new motion blur model has been built to acquire data for “motion-from-smear”. Experimental results on images with both simulated smear and real smear, using the authors' “motion-from-smear” algorithm as well as a conventional motion estimation technique, are provided. The authors also show that temporal aliasing does not affect “motion-from-smear” to the same degree as it does algorithms that use displacement as a cue. “Motion-from-smear” provides an additional tool for motion estimation and effectively complements the existing techniques when apparent motion smear is present     

Fast and robust absolute camera pose estimation with known focal length

Some 3D computer vision techniques such as structure from motion (SFM) and augmented reality (AR) depend on a specific perspective-n-point (PnP) algorithm to estimate the absolute camera pose. However, existing PnP algorithms are difficult to achieve a good balance between accuracy and efficiency, and most of them do not make full use of the internal camera information such as focal length. In order to attack these drawbacks, we propose a fast and robust PnP (FRPnP) method to calculate the absolute camera pose for 3D compute vision. In the proposed FRPnP method, we firstly formulate the PnP problem as the optimization problem in the null space that can avoid the effects of the depth of each 3D point. Secondly, we can easily get the solution by the direct manner using singular value decomposition. Finally, the accurate information of camera pose can be obtained by optimization strategy. We explore four ways to evaluate the proposed FRPnP algorithm with synthetic dataset, real images, and apply it in the AR and SFM system. Experimental results show that the proposed FRPnP method can obtain the best balance between computational cost and precision, and clearly outperforms the state-of-the-art PnP methods.

     

Extracting 3D facial animation parameters from multiview video clips

We propose an accurate and inexpensive procedure that estimates 3D facial motion parameters from mirror-reflected multiview video clips. We place two planar mirrors near a subject's cheeks and use a single camera to simultaneously capture a marker's front and side view images. We also propose a novel closed-form linear algorithm to reconstruct 3D positions from real versus mirrored point correspondences in an uncalibrated environment. Our computer simulations reveal that exploiting mirrors' various reflective properties yields a more robust, accurate, and simpler 3D position estimation approach than general-purpose stereo vision methods that use a linear approach or maximum-likelihood optimization. Our experiments show a root mean square (RMS) error of less than 2 mm in 3D space with only 20-point correspondences. For semiautomatic 3D motion tracking, we use an adaptive Kalman predictor and filter to improve stability and infer the occluded markers' position. Our approach tracks more than 50 markers on a subject's face and lips from 30-frame-per-second video clips. We've applied the facial motion parameters estimated from the proposed method to our facial animation system.     

A new robust algorithmic for multi-camera calibration with a 1D object under general motions without prior knowledge of any camera intrinsic parameter

In computer vision, camera calibration is a necessary process when the retrieval of information such as angles and distances is required. This paper addresses the multi-camera calibration problem with a single dimension calibration pattern under general motions. Currently, the known algorithms for solving this problem are based on the estimation of vanishing points. However, this estimate is very susceptible to noise, making the methods unsuitable for practical applications. Instead, this paper presents a new calibration algorithm, where the cameras are divided into binocular sets. The fundamental matrix of each binocular set is then estimated, allowing to perform a projective calibration of each camera. Then, the calibration is updated for the Euclidean space, ending the process. The calibration is possible without imposing any restrictions on the movement of the pattern and without any prior information about the cameras or motion. Experiments on synthetic and real images validate the new method and show that its accuracy makes it suitable also for practical applications.     

Vision‐Based Tracking and Position Estimation of Moving Targets for Unmanned Helicopter Systems

The primary goal of this study is to track a ground‐moving target using a machine‐vision system installed on an unmanned helicopter, and to estimate its position if the target becomes unobservable. The machine‐vision system is accomplished using real‐time color images obtained from a charge‐coupled device (CCD) camera mounted on a computer‐controlled gimbaled system that can pitch and yaw. To avoid real‐time image‐tracking failure resulting from a moving target becoming concealed, the Kalman filtering technique is applied to predict the target's follow‐on position, so that the camera can continuously track the target. The entire system is initially tested on the ground and then mounted on a helicopter for in‐flight testing. The following three cases are shown in the flight tests: (1) an uncovered static target; (2) a moving visible target; and (3) a target that moves in a straight line at a constant speed and becomes temporarily concealed. The vision‐based tracking system with the developed algorithm is successfully applied in all three cases.     

An intensity-based cooperative bidirectional stereo matching with simultaneous detection of discontinuities and occlusions

This paper presents a new intensity-based stereo algorithm using cooperative bidirectional matching with a hierarchical multilevel structure. Based on a new model of piecewise smooth depth fields and the consistency constraint, the algorithm is able to estimate the 3-D structure and detect its discontinuities and the occlusion reliably with low computational costs. In order to find the global optimal estimates, we utilize a multiresolution two-stage algorithm minimizing nonconvex cost functions, which is equivalent to the MAP estimation. This basic framework computing the 3-D structure from binocular stereo images has been extended to the trinocular stereo vision for a further improvement of the performance. A few examples for the binocular and trinocular stereo problems are given to illustrate the performance of the new algorithms.     

On the Consistency of Instantaneous Rigid Motion Estimation

Instantaneous camera motion estimation is an important research topic in computer vision. Although in theory more than five points uniquely determine the solution in an ideal situation, in practice one can usually obtain better estimates by using more image velocity measurements because of the noise present in the velocity measurements. However, the usefulness of using a large number of observations has never been analyzed in detail. In this paper, we formulate this problem in the statistical estimation framework. We show that under certain noise models, consistency of motion estimation can be established: that is, arbitrarily accurate estimates of motion parameters are possible with more and more observations. This claim does not simply follow from the general consistency result for maximum likelihood estimates. Some experiments will be provided to verify our theory. Our analysis and experiments also indicate that many previously proposed algorithms are inconsistent under even very simple noise models.     

基于联合双边滤波器的Kinect深度图像滤波算法

针对Kinect镜头采集的深度图像一般有噪声和黑洞现象,直接应用于人体动作跟踪和识别等系统中效果差的问题,提出一种基于联合双边滤波器的深度图像滤波算法。算法利用联合双边滤波原理,将Kinect镜头同一时刻采集的深度图像和彩色图像作为输入,首先,用高斯核函数计算出深度图像的空间距离权值和RGB彩色图像的灰度权值;然后,将这两个权值相乘得到联合滤波权值,并利用快速高斯变换替换高斯核函数,设计出联合双边滤波器;最后,用此滤波器的滤波结果与噪声图像进行卷积运算实现Kinect深度图像滤波。实验结果表明,所提算法应用在人体动作识别和跟踪系统后,可显著提高在背景复杂场景中的抗噪能力,识别正确率提高17.3%,同时所提算法的平均耗时为371ms,远低于同类算法。所提算法保持了联合双边滤波平滑保边的优点,由于引入彩色图像作为引导图像,去噪的同时也能对黑洞进行修补,因此该算法在Kinect深度图像上的去噪和修复效果优于经典的双边滤波算法和联合双边滤波算法,且实时性强。     

A study of the Kalman filter as a state estimator of deterministic and stochastic systems

It is shown that the optimality of the Kalman filter prevents its application to estimate the state of deterministic; systems but that if the gain is slightly modified a deterministic filter is possible. Such a filter is subsequently used to explain some of the difficulties encountered in Kalman filter computations, from which it transpires that much Kalman filtering is essentially an application of a deterministic filter to stochastic problems. To reduce the order of the algorithm the concept of observer robustness is incorporated into the subsequent development of the deterministic filter. Exactly equivalent discrete- and continuous-time algorithms are derived and used for a new treatment of the problem of obtaining estimates of the state variables of a stochastic system when the measurements are free of noise.