Robust structure and motion recovery for monocular vision systems with noisy measurements

This study proposes a novel complete-order nonlinear structure and motion observer for monocular vision systems subjected to significant measurement noise. In contrast with previous studies that assume noise-free measurements, and require prior knowledge of either the relative motion of the camera or scene geometry, the proposed scheme assumes a single component of linear velocity as known. Under a persistency of excitation condition, the observer then relies on filtered estimates of optical flow to yield exponentially convergent estimates of the unknown motion parameters and feature depth that converge to a uniform, ultimate bound in the presence of measurement noise. The unknown linear and angular velocities are assumed to be generated using an imperfectly known model that incorporates a bounded uncertainty, and optical flow estimation is accomplished using a robust differentiator that is based on the sliding-mode technique. Numerical results are used to validate and demonstrate superior observer performance compared to an alternative leading design in the presence of model uncertainty and measurement noise.     

基于扩展卡尔曼滤波的船舶横向运动扰动估计

建立了船舶横向运动状态方程和测量方程,利用扩展卡尔曼滤波方法对海浪扰动下的船舶横向运动的扰动力和力矩作出估计。仿真结果表明,扩展卡尔曼滤波法比有色卡尔曼滤波法估计效果更优。     

An Optical Flow-Based Solution to the Problem of Range Identification in Perspective Vision Systems

A classical problem in machine vision is the range identification of an object moving in three-dimensional space from the two-dimensional image sequence obtained with a monocular camera. This study presents a novel reduced-order optical flow-based nonlinear observer that renders the proposed scheme suitable for depth estimation applications in both well-structured and unstructured environments. In this study, a globally exponentially stable observer is synthesized, where optical flow estimates are derived from tracking feature trajectory on the image plane over successive camera frames, to yield asymptotic estimates of feature depth at a desired convergence rate. Furthermore, the observer is shown to be finite-gain \(\mathcal {L}_{p}\) stable ?p∈[1,∞] in the presence of exogenous disturbance influencing camera motion, and is applicable to a wider class of perspective systems than those considered by alternative designs. The observer requires minor apriori system information for convergence, and the convergence condition arises in a natural manner with an apparently intuitive interpretation. Numerical and experimental studies are used to validate and demonstrate robust observer performance in the presence of significant measurement noise.     

Visually guided ranging from observations of points, lines and curves via an identifier based nonlinear observer

In this paper we consider the problem of estimating the range information of features on an affine plane in by observing its image with the aid of a CCD camera, wherein we assume that the camera is undergoing a known motion. The features considered are points, lines and planar curves located on planar surfaces of static objects. The dynamics of the moving projections of the features on the image plane have been described as a suitable differential equation on an appropriate feature space. This dynamics is used to estimate feature parameters from which the range information is readily available. In this paper the proposed identification has been carried out via a newly introduced identifier based observer. Performance of the observer has been studied via simulation.     

Invariant Observer Design of Attitude and Heading Reference System

An attitude and heading reference system (AHRS) is a nonlinear state estimator unit for computing orientation in 3D space. This paper designs an AHRS using three approaches: an invariant observer, an invariant extended Kalman filter (IEKF), and a conventional extended Kalman filter (EKF). The three designs are validated in experiment versus a ground truth, demonstrating the practical interest of the invariant observer methodology and the advantage of the IEKF over the EKF under model uncertainty.     

Motion from occlusions

In computer vision, occlusions are almost always seen as undesirable singularities that pose difficult challenges to image motion analysis problems, such as optic flow computation, motion segmentation, disparity estimation, or egomotion estimation. However, it is well known that occlusions are extremely powerful cues for depth or motion perception, and could be used to improve those methods.

In this paper, we propose to recover camera motion information based uniquely on occlusions, by observing two specially useful properties: occlusions are independent of the camera rotation, and reveal direct information about the camera translation.

We assume a monocular observer, undergoing general rotational and translational motion in a static environment. We present a formal model for occlusion points and develop a method suitable for occlusion detection. Through the classification and analysis of the detected occlusion points, we show how to retrieve information about the camera translation (FOE). Experiments with real images are presented and discussed in the paper.     

Range identification for perspective vision systems

In this note, a new observer is developed to determine range information (and, hence, the three-dimensional (3-D) coordinates) of an object feature moving with affine motion dynamics (or the more general Ricatti motion dynamics) with known motion parameters. The unmeasurable range information is determined from a single camera provided an observability condition is satisfied that has physical significance. To develop the observer, the perspective system is expressed in terms of the nonlinear feature dynamics. The structure of the proposed observer is inspired by recent disturbance observer results. The proposed technique facilitates a Lyapunov-based analysis that is less complex than the sliding-mode based analysis derived for recent observer designs. The analysis demonstrates that the 3-D task-space coordinates of the feature point can be asymptotically identified. Simulation results are provided that illustrate the performance of the observer in the presence of noise.     

电子稳像中基于运动分类的全局运动估计算法

提出一种基于运动分类的全局运动估计算法.首先,分区提取图像中的鲁棒Harris特征点,并采用特征窗匹配思路,提高匹配速度;其次,对运动矢量在平移、旋转和缩放模式下的统计特性进行分析,提出运动类型快速判定方法,并验证特征点的有效性;再次,将有效点对代入运动方程,求取全局运动参数;最后,结合Kalman滤波来补偿当前帧实现视频稳像.实验结果表明,该算法能够处理含摄像机扫描和抖动的复杂场景,检测误差小于0.5像素,且达到实时处理.     

Recursive estimation of 3D motion and surface structure from local affine flow parameters

A recursive structure from motion algorithm based on optical flow measurements taken from an image sequence is described. It provides estimates of surface normal in addition to 3D motion and depth. The measurements are affine motion parameters which approximate the local flow fields associated with near-planar surface patches in the scene. These are integrated over time to give estimates of the 3D parameters using an extended Kalman filter. This also estimates the camera focal length and, so, the 3D estimates are metric. The use of parametric measurements means that the algorithm is computationally less demanding than previous optical flow approaches and the recursive filter builds in a degree of noise robustness. Results of experiments on synthetic and real image sequences demonstrate that the algorithm performs well.     

Passive ranging using a moving camera

This article addresses the problem of determining the 3-dimensional locations of salient points in the environment of a moving camera based on a monocular image sequence obtained by the camera. The camera's translational and rotational velocities are assumed to be known approximately via inertial sensors. The motion of the camera is represented by a constant velocity model. Salient points in the image sequence are extracted using Gabor wavelets and tracked using labeled graph matching. The 3-D positions of the selected environmental points relative to the camera are then estimated recursively using an extended Kalman filter (EKF), after initialization by two-frame motion stereo. The motion parameters of the camera are also refined simultaneously. Experimental results on real data are given. © 1992 John Wiley & Sons, Inc.     

Recovery of the 3-D location and motion of a rigid object through camera image (An Extended Kalman Filter Approach)

This paper deals with the problem of locating a rigid object and estimating its motion in three dimensions. This involves determining the position and orientation of the object at each instant when an image is captured by a camera, and recovering the motion of the object between consecutive frames.In the implementation scheme used here, a sequence of camera images, digitized at the sample instants, is used as the initial input data. Measurements are made of the locations of certain features (e.g., maximum curvature points of an image contour, corners, edges, etc.) on the 2-D images. To measure the feature locations a matching algorithm is used, which produces correspondences between the features in the image and the object.Using the measured feature locations on the image, an algorithm is developed to solve the location and motion problem. The algorithm is an extended Kalman filter modeled for this application.Department of Electrical Engineering and Alberta Center for Machine Intelligence and Robotics, University of Alberta     

一种鲁棒的基于两个特征点的深度估计方法

This paper presents a novel depth estimation method based on feature points. Two points are selected arbitrarily from an object and their distance in the space is assumed to be known.The proposed technique can estimate simultaneously their depths according to two images taken before and after a camera moves and the motion parameters of the camera may be unknown. In addition, this paper analyzes the ways to enhance the precision of the estimated depths and presents a feature point image coordinates search algorithm to increase the robustness of the proposed method.The search algorithm can find automatically more accurate image coordinates of the feature points based on their detected image coordinates. Experimental results demonstrate the efficiency of the presented method.     

A Theory of Specular Surface Geometry

A theoretical framework is introduced for the perception of specular surface geometry. When an observer moves in three-dimensional space, real scene features such as surface markings remain stationary with respect to the surfaces they belong to. In contrast, a virtual feature which is the specular reflection of a real feature, travels on the surface. Based on the notion of caustics, a feature classification algorithm is developed that distinguishes real and virtual features from their image trajectories that result from observer motion. Next, using support functions of curves, a closed-form relation is derived between the image trajectory of a virtual feature and the geometry of the specular surface it travels on. It is shown that, in the 2D case, where camera motion and the surface profile are coplanar, the profile is uniquely recovered by tracking just two unknown virtual features. Finally, these results are generalized to the case of arbitrary 3D surface profiles that are traveled by virtual features when camera motion is not confined to a plane. This generalization includes a number of mathematical results that substantially enhance the present understanding of specular surface geometry. An algorithm is developed that uniquely recovers 3D surface profiles using a single virtual feature tracked from the occluding boundary of the object. All theoretical derivations and proposed algorithms are substantiated by experiments.     

Early‐ and late‐lumping observer designs for long hydraulic pipelines: Application to pumped‐storage power plants

Pumped‐storage power plants typically feature very long hydraulic pipelines, which can be modeled by a set of partial differential equations. The estimation of the pressure and volumetric flow along the pipes is an important task for the operation of such a plant. Therefore, this work compares different early‐ and late‐lumping–based observer designs for this system. Two late‐lumping observers, ie, a Lyapunov‐based design and an observer using the backstepping design method, are examined. The Lyapunov‐based approach uses a simple boundary correction to stabilize the estimation error dynamics. In contrast, the backstepping‐based approach allows utilizing additional in‐domain correction to obtain a faster rate of convergence. For the implementation of these distributed‐parameter observers, the spectral element method as a flexible and computationally efficient discretization method is introduced. It is shown that, compared with that of the Lyapunov‐based design, the discretization of the backstepping‐based design requires additional spatial grid points for the accurate approximation of its feedback gains. For the early‐lumping approach, the spectral element method is used to approximate the model equations by a system of differential equations. Based on this approximation, an extended Kalman filter is designed. All observer designs are validated and compared for a representative test case.     

Adaptive nonlinear observer for state and unknown parameter estimation in noisy systems

This paper proposes a novel adaptive observer for Lipschitz nonlinear systems and dissipative nonlinear systems in the presence of disturbances and sensor noise. The observer is based on an H_∞ observer that can estimate both the system states and unknown parameters by minimising a cost function consisting of the sum of the square integrals of the estimation errors in the states and unknown parameters. The paper presents necessary and sufficient conditions for the existence of the observer, and the equations for determining observer gains are formulated as linear matrix inequalities (LMIs) that can be solved offline using commercially available LMI solvers. The observer design has also been extended to the case of time-varying unknown parameters. The use of the observer is demonstrated through illustrative examples and the performance is compared with extended Kalman filtering. Compared to previous results on nonlinear observers, the proposed observer is more computationally efficient, and guarantees state and parameter estimation for two very broad classes of nonlinear systems (Lipschitz and dissipative nonlinear systems) in the presence of input disturbances and sensor noise. In addition, the proposed observer does not require online computation of the observer gain.     

Rigid body segmentation and shape description from dense opticalflow under weak perspective

We present an algorithm for identifying and tracking independently moving rigid objects from optical flow. Some previous attempts at segmentation via optical flow have focused on finding discontinuities in the flow field. While discontinuities do indicate a change in scene depth, they do not in general signal a boundary between two separate objects. The proposed method uses the fact that each independently moving object has a unique epipolar constraint associated with its motion. Thus motion discontinuities based on self-occlusion can be distinguished from those due to separate objects. The use of epipolar geometry allows for the determination of individual motion parameters for each object as well as the recovery of relative depth for each point on the object. The algorithm assumes an affine camera where perspective effects are limited to changes in overall scale. No camera calibration parameters are required. A Kalman filter based approach is used for tracking motion parameters with time     

视觉SLAM与构建地图方法概述

概述了基于视觉SLAM经典框架的五个组成部分,重点分析了特征点法的简单原理及特性、非线性系统和扩展卡尔曼滤波以及回环检测的词袋模型.最后展望了未来基于深度相机的SLAM研究和发展方向.     

Image feature command generation of contour following tasks for SCARA robots employing Image-Based Visual Servoing—A PH-spline approach

Recently, visual servoing has been widely employed in industrial robots and has become an invaluable asset to enhance the functionality of the robot. However, the issue of image feature command generation in a visual servoing task receives little attention. In a contour following task that adopts Image-Based Visual Servoing (IBVS), it is crucial to perform motion planning on the desired image trajectory. Without proper motion planning, not only may the discrepancy between the target position and the current position on the image plane not converge, but also the flexibility of exploiting visual servoing for applications such as contour following will be limited. In order to cope with the aforementioned problem, this paper proposes a PH-spline based motion planning approach for systems that adopt IBVS. In particular, the exterior contour of an object is represented by a PH quantic spline. With proper acceleration/deceleration motion planning, a PH quantic spline interpolator is constructed to generate desired image feature commands so that IBVS can be applied to handle contour following problems of an object without a known geometric model. Furthermore, this paper also develops a depth estimation algorithm for the eye-to-hand camera structure, providing a convenient way to estimate the depth value that is essential in computing image Jacobian. Experimental results of several contour following tasks verify the effectiveness of the proposed approach.