An Analytically Stable Structure and Motion Observer Based on Monocular Vision

This paper presents a novel nonlinear sliding-mode differentiator-based complete-order observer for structure and motion identification with a calibrated monocular camera. In comparison with earlier work that requires prior knowledge of either the Euclidean geometry of the observed object or the linear acceleration of the camera and is restricted to establishing stability and convergence from image-plane measurements of a single tracked feature, the proposed scheme assumes partial velocity state feedback to asymptotically identify the true-scale Euclidean coordinates of numerous observed object features and the unknown motion parameters. The dynamics of the motion parameters are assumed to be described by a model with unknown parameters that incorporates a bounded uncertainty, and a Lyapunov analysis is provided to prove that the observer yields exponentially convergent estimates that converge to a uniform ultimate bound under a generic persistency of excitation condition. Numerical and experimental results are obtained that demonstrate the robust performance of the current scheme in the presence of model error and measurement noise.     

Visually guided ranging from observations of points, lines and curves via an identifier based nonlinear observer

In this paper we consider the problem of estimating the range information of features on an affine plane in by observing its image with the aid of a CCD camera, wherein we assume that the camera is undergoing a known motion. The features considered are points, lines and planar curves located on planar surfaces of static objects. The dynamics of the moving projections of the features on the image plane have been described as a suitable differential equation on an appropriate feature space. This dynamics is used to estimate feature parameters from which the range information is readily available. In this paper the proposed identification has been carried out via a newly introduced identifier based observer. Performance of the observer has been studied via simulation.     

Range identification for nonlinear parameterizable paracatadioptric systems

In this paper, a new range identification technique for a calibrated paracatadioptric system mounted on a moving platform is developed to recover the range information and the three-dimensional (3D) Euclidean coordinates of a static object feature. The position of the moving platform is assumed to be measurable. To identify the unknown range, first, a function of the projected pixel coordinates is related to the unknown 3D Euclidean coordinates of an object feature. This function is nonlinearly parameterized (i.e., the unknown parameters appear nonlinearly in the parameterized model). An adaptive estimator based on a min-max algorithm is then designed to estimate the unknown 3D Euclidean coordinates of an object feature relative to a fixed reference frame which facilitates the identification of range. A Lyapunov-type stability analysis is used to show that the developed estimator provides an estimation of the unknown parameters within a desired precision. Numerical simulation results are presented to illustrate the effectiveness of the proposed range estimation technique.     

视景仿真的关键技术

针对视景仿真开发中的几个常见问题,提出了解决方案。包括：利用回调函数将OpencL代码嵌入到Vega应用程序中,实现Vega中未能提供的图形功能;分析了视景坐标系与模型坐标系之间的对应转换关系,给出两种常用的转换公式;概述了通用视点方式的设计方法,通过开发自定义运动模式实现了以输入设备控制视点的方法。     

Nonlinear observers for Lipschitz continuous systems with inputs

This paper considers the state observation problem for nonlinear dynamical systems. The proposed framework is a direct generalization of a method introduced in a recent paper for autonomous system. Its characteristic feature is that the dynamic part of the observer is linear and, as a consequence, that convergence takes place globally in the observer coordinates. The observer is completed by a static nonlinearity which maps the observer state in the original state space. An associated observation mapping is introduced and is interpreted in terms of an orthonormal expansion of the input and the output with respect to a certain basis in a suitable Hilbert space. It is shown that, by choosing the observer dimension properly, an observer with arbitrary small asymptotic observation error is obtained, provided that some compactness properties for the subset to be observed and the set of input signals hold. Under a stronger condition, the finite complexity property, an exact observer is achieved. Finally, an integral formula representation for the observer nonlinearity is given.     

Fusion of range camera and photogrammetry: a systematic procedure for improving 3-D models metric accuracy

The generation of three-dimensional (3-D) digital models produced by optical technologies in some cases involves metric errors. This happens when small high-resolution 3-D images are assembled together in order to model a large object. In some applications, as for example 3-D modeling of Cultural Heritage, the problem of metric accuracy is a major issue and no methods are currently available for enhancing it. The authors present a procedure by which the metric reliability of the 3-D model, obtained through iterative alignments of many range maps, can be guaranteed to a known acceptable level. The goal is the integration of the 3-D range camera system with a close range digital photogrammetry technique. The basic idea is to generate a global coordinate system determined by the digital photogrammetric procedure, measuring the spatial coordinates of optical targets placed around the object to be modeled. Such coordinates, set as reference points, allow the proper rigid motion of few key range maps, including a portion of the targets, in the global reference system defined by photogrammetry. The other 3-D images are normally aligned around these locked images with usual iterative algorithms. Experimental results on an anthropomorphic test object, comparing the conventional and the proposed alignment method, are finally reported.     

Vehicle state estimation for anti-lock control with nonlinear observer

Vehicle state estimation during anti-lock braking is considered. A novel nonlinear observer based on a vehicle dynamics model and a simplified Pacejka tire model is introduced in order to provide estimates of longitudinal and lateral vehicle velocities and the tire-road friction coefficient for vehicle safety control systems, specifically anti-lock braking control. The approach differs from previous work on vehicle state estimation in two main respects. The first is the introduction of a switched nonlinear observer in order to deal with the fact that in some driving situations the information provided by the sensor is not sufficient to carry out state estimation (i.e., not all states are observable). This is shown through an observability analysis. The second contribution is the introduction of tire-road friction estimation depending on vehicle longitudinal motion. Stability properties of the observer are analyzed using a Lyapunov function based method. Practical applicability of the proposed nonlinear observer is shown by means of experimental results.     

Biological cell injection visual and haptic interface

This paper presents a new three-dimensional (3-D) biomicromanipulation system for biological objects such as embryos, cells or oocytes. As the cell is very small, kept in liquid and observed through a microscope, 2-D visual feedback makes accurate manipulation in the 3-D world difficult. To improve the manipulation work, we proposed an intelligent human–machine interface. The 3-D visual information is provided to the operator through a 3-D reconstruction method using vision-based tracking deformations of the cell embryo. In order to perform stable microinjection tasks, the operator needs force feedback and haptic assistance during penetration of the cell envelop — the chorion. Thus, realistic haptic rendering techniques have been implemented to validate stable insertion of a micropipette in a living cell. The proposed human–machine user's interface allows real-time realistic visual and haptic control strategies for constrained motion in image coordinates, virtual haptic rendering to constrain the path of insertion and removal in the 3-D scene or to avoid cell destruction by adequately controlling position, velocity and force parameters. Experiments showed that the virtualized reality interface acts as a tool for total guidance and assistance during microinjection tasks.     

Statistical analysis of inherent ambiguities in recovering 3-Dmotion from a noisy flow field

The inherent ambiguities in recovering 3-D motion information from a single optical flow field are studied using a statistical model. The ambiguities are quantified using the Cramer-Rao lower bound. As a special case, the performance bound for the motion of 3-D rigid planar surfaces is studied in detail. The dependence of the bound on factors such as the underlying motion, surface position, surface orientation, field of view, and density of available pixels are derived as closed-form expressions. A subset of the results support S. Adiv's (1989) analysis of the inherent ambiguities of motion parameters. For the general motion of an arbitrary surface. It is shown that the aperture problem in computing the optical flow restricts the nontrivial information about the 3-D motion to a sparse set of pixels at which both components of the flow velocity are observable. Computer simulations are used to study the dependence of the inherent ambiguities on the underlying motion, the field of view, and the number of feature points for the motion in front of a nonplanar environment     

Asymptotic observers for non-linear systems†

Observer design for non-linear systems is discussed. Some recent approaches are based on state and output change of coordinates to transform a non-linear system into a particular observer form, from which an asymptotic observer can be designed ensuring the asymptotic stability of the error dynamics in the new coordinates. In this paper, the stability properties of the error dynamics are studied in the original coordinates. With some examples, it is shown how the asymptotic stability in the new coordinates does not imply, in general, the asymptotic stability in the original ones. Some general results are stated and proved to guarantee the asymptotic stability of the error dynamics in the original coordinates.     

The feasibility of motion and structure from noisy time-varying image velocity information

This research addresses the problem of noise sensitivity inherent in motion and structure algorithms. The motion and structure paradigm is a two-step process. First, we measure image velocities and, perhaps, their spatial and temporal derivatives, are obtained from time-varying image intensity data and second, we use these data to compute the motion of a moving monocular observer in a stationary environment under perspective projection, relative to a single 3-D planar surface. The first contribution of this article is an algorithm that uses time-varying image velocity information to compute the observer's translation and rotation and the normalized surface gradient of the 3-D planar surface. The use of time-varying image velocity information is an important tool in obtaining a more robust motion and structure calculation. The second contribution of this article is an extensive error analysis of the motion and structure problem. Any motion and structure algorithm that uses image velocity information as its input should exhibit error sensitivity behavior compatible with the results reported here. We perform an average and worst case error analysis for four types of image velocity information: full and normal image velocities and full and normal sets of image velocity and its derivatives. (These derivatives are simply the coefficients of a truncated Taylor series expansion about some point in space and time.) The main issues we address here are: just how sensitive is a motion and structure computation in the presence of noisy input, or alternately, how accurate must our image velocity information be, how much and what type of input data is needed, and under what circumstances is motion and structure feasible? That is, when can we be sure that a motion and structure computation will produce usable results? We base our answers on a numerical error analysis we conduct for a large number of motions.     

Adaptive Visual Servoing Using Point and Line Features With an Uncalibrated Eye-in-Hand Camera

This paper presents a novel approach for image-based visual servoing of a robot manipulator with an eye-in-hand camera when the camera parameters are not calibrated and the 3-D coordinates of the features are not known. Both point and line features are considered. This paper extends the concept of depth-independent interaction (or image Jacobian) matrix, developed in earlier work for visual servoing using point features and fixed cameras, to the problem using eye-in-hand cameras and point and line features. By using the depth-independent interaction matrix, it is possible to linearly parameterize, by the unknown camera parameters and the unknown coordinates of the features, the closed-loop dynamics of the system. A new algorithm is developed to estimate unknown parameters online by combining the Slotine–Li method with the idea of structure from motion in computer vision. By minimizing the errors between the real and estimated projections of the feature on multiple images captured during motion of the robot, this new adaptive algorithm can guarantee the convergence of the estimated parameters to the real values up to a scale. On the basis of the nonlinear robot dynamics, we proved asymptotic convergence of the image errors by the Lyapunov theory. Experiments have been conducted to demonstrate the performance of the proposed controller.      

基于RBF神经网络的复杂背景下的运动目标检测

由于运动摄像机的存在使得复杂背蒂下的运动目标检测问题更加复杂,根据场景中目标与背景具有不同的运动、任意场景可以分成不同的运动区域这一基拳事实,提出一种新的基于RBF神经网络的运动目标检测算法。运动补偿后求参考帧与补偿后的当前帧之间的光流,联合当前像素坐标及其灰度值得到五雏特征向量作为RBF网络的输入,RBF网络学习算法通过最小化由Bayesian理论和能量最小化理论导出的损失函数实现。学习矢量量化方法修正网络的中心,收敛后网络的输出就是运动目标区域。试验结果证明了算法的有效性。     

点状特征柔性物体三维运动捕获方法

针对具有点状特征的柔性物体,提出了一种三维运动捕获方法.首先,该方法利用两个标定的高速摄像机拍摄柔性物体的运动视频,并对图像进行立体校正;然后,采用DOG （Difference Of Gaussian）算法获取点状特征的位置,并提取特征点极值;其次,在一定范围的窗口上搜索匹配对,匹配左右图像的特征点;再次,通过三角测量法进行三维重建;最后,利用搜索策略进行时间序列上的匹配,实现动态柔性物体的三维运动捕获,并计算空间坐标、速度、加速度参数.实验结果表明,相比于采用sift算法匹配特征点捕获柔性运动物体的方法,本方法精度更高.     

Egomotion analysis based on the Frenet-Serret motion model

In this paper we propose a new model,Frenet-Serret motion, for the motion of an observer in a stationary environment. This model relates the motion parameters of the observer to the curvature and torsion of the path along which the observer moves. Screw-motion equations for Frenet-Serret motion are derived and employed for geometrical analysis of the motion. Normal flow is used to derive constraints on the rotational and translational velocity of the observer and to compute egomotion by intersecting these constraints in the manner proposed in (Duri and Aloimonos 1991) The accuracy of egomotion estimation is analyzed for different combinations of observer motion and feature distance. We explain the advantages of controlling feature distance to analyze egomotion and derive the constraints on depth which make either rotation or translation dominant in the perceived normal flow field. The results of experiments on real image sequences are presented.The support of the Air Force Office of Scientific Research under Grant F49620-93-1-0039 is gratefully acknowledged.     

Visual servoing of mobile robots for posture stabilization: from theory to experiments

On the basis of the kinematic model of a unicycle mobile robot in polar coordinates, an adaptive visual servoing strategy is proposed to regulate the mobile robot to its desired pose. By regarding the unknown depth as model uncertainty, the system error vector can be chosen as measurable signals that are reconstructed by a motion estimation technique. Then, an adaptive controller is carefully designed along with a parameter updating mechanism to compensate for the unknown depth information online. On the basis of Lyapunov techniques and LaSalle's invariance principle, rigorous stability analysis is conducted. Because the control law is elegantly designed on the basis of the polar‐coordinate‐based representation of error dynamics, the consequent maneuver behavior is natural, and the resulting path is short. Experimental results are provided to verify the performance of the proposed approach. Copyright © 2013 John Wiley & Sons, Ltd.     

3-D motion estimation using a sequence of noisy stereo images:models, estimation, and uniqueness results

A kinematic model-based approach for the estimation of 3-D motion parameters from a sequence of noisy stereo images is discussed. The approach is based on representing the constant acceleration translational motion and constant precession rotational motion in the form of a bilinear state-space model using standard rectilinear states for translation and quaternions for rotation. Closed-form solutions of the state transition equations are obtained to propagate the quaternions. The measurements are noisy perturbations of 3-D feature points represented in an inertial coordinate system. It is assumed that the 3-D feature points are extracted from the stereo images and matched over the frames. Owing to the nonlinearity in the state model, nonlinear filters are designed for the estimation of motion parameters. Simulation results are included. The Cramer-Rao performance bounds for motion parameter estimates are computed. A constructive proof for the uniqueness of motion parameters is given. It is shown that with uniform sampling in time, three noncollinear feature points in five consecutive binocular image pairs contain all the spatial and temporal information. Both nondegenerate and degenerate motions are analyzed. A deterministic algorithm to recover motion parameters from a stereo image sequence is summarized from the constructive proof     

微小无人直升机自适应视觉伺服

针对摄像机未标定和特征点坐标未知的情况, 本文提出一种新颖的基于图像的无人直升机自适应视觉伺服方法. 控制器是基于反推法设计的, 但是和已有的基于反推法的视觉伺服不同的是, 它利用与深度无关矩阵将图像误差映射到执行器空间, 从而可以避免估计特征点的深度. 这种设计方法可以线性化未知的摄像机参数和特征点坐标, 所以能方便地设计自适应算法来在线估计这些未知参数, 同时为了保证图像误差收敛和避免估计参数收敛至零解而引入了两个势函数. 利用Lyapunov方法证明了基于非线性动力学的控制器的稳定性, 并给出了仿真验证.