Uncalibrated obstacle detection using normal flow

This paper addresses the problem of obstacle detection for mobile robots. The visual information provided by a single on-board camera is used as input. We assume that the robot is moving on a planar pavement, and any point lying outside this plane is treated as an obstacle. We address the problem of obstacle detection by exploiting the geometric arrangement between the robot, the camera, and the scene. During an initialization stage, we estimate an inverse perspective transformation that maps the image plane onto the horizontal plane. During normal operation, the normal flow is computed and inversely projected onto the horizontal plane. This simplifies the resultant flow pattern, and fast tests can be used to detect obstacles. A salient feature of our method is that only the normal flow information, or first order time-and-space image derivatives, is used, and thus we cope with the aperture problem. Another important issue is that, contrasting with other methods, the vehicle motion and intrinsic and extrinsic parameters of the camera need not be known or calibrated. Both translational and rotational motion can be dealt with. We present motion estimation results on synthetic and real-image data. A real-time version implemented on a mobile robot, is described.     

Estimations of previewed road curvatures and vehicular motion by a vision-based data fusion scheme

In this study, a new framework of vision-based estimation is developed using some data fusion schemes to obtain previewed road curvatures and vehicular motion states based on the scene viewed from an in-vehicle camera. The previewed curvatures are necessary for the guidance of an automatically steering vehicle, and the desired vehicular motion variables, including lateral deviation, heading angle, yaw rate, and sideslip angle, are also required for proper control of the vehicular lateral motion via steering. In this framework, physical relationships of previewed curvatures among consecutive images, motion variables in terms of image features searched at various levels in the image plane, and dynamic correlation among vehicular motion variables are derived as bases of data fusion to enhance the accuracy of estimation. The vision-based measurement errors are analyzed to determine the fusion gains based on the technique of a Kalman filter such that the measurements from the image plane and predictions of physical models can be properly integrated to obtain reliable estimations. Off-line experimental works using real road scenes are performed to verify the whole framework for image sensing.     

Visual Surveillance for Moving Vehicles

An overview is given of a vision system for locating, recognising and tracking multiple vehicles, using an image sequence taken by a single camera mounted on a moving vehicle. The camera motion is estimated by matching features on the ground plane from one image to the next. Vehicle detection and hypothesis generation are performed using template correlation and a 3D wire frame model of the vehicle is fitted to the image. Once detected and identified, vehicles are tracked using dynamic filtering. A separate batch mode filter obtains the 3D trajectories of nearby vehicles over an extended time. Results are shown for a motorway image sequence.     

Image Sequence Stabilization in Real Time

This paper describes a method ofstabilizingimage sequences obtained by a camera carried by a ground vehicle. The motion of the vehicle can usually be regarded as consisting of a desired smooth motion combined with an undesired non-smooth motion that includes impulsive or high-frequency components. The goal of the stabilization process is to correct the images so that they are approximately the same as the images that would have been obtained if the motion of the vehicle had been smooth.We analyse the smooth and non-smooth motions of a ground vehicle and show that only the rotational components of the non-smooth motion have significant perturbing effects on the images. We show how to identify image points at which rotational image flow is dominant, and how to use such points to estimate the vehicle's rotation. Finally, we describe an algorithm that fits smooth (ideally, piecewise constant) rotational motions to these estimates; the residual rotational motion can then be used to correct the images. We have obtained good results for several image sequences obtained from a camera carried by a ground vehicle moving across bumpy terrain.     

Monocular obstacle detection using reciprocal-polar rectification

Our obstacle detection method is applicable to deliberative translation motion of a mobile robot and, in such motion, the epipole of each image of an image pair is coincident and termed the focus of expansion (FOE). We present an accurate method for computing the FOE and then we use this to apply a novel rectification to each image, called a reciprocal-polar (RP) rectification. When robot translation is parallel to the ground, as with a mobile robot, ground plane image motion in RP-space is a pure shift along an RP image scan line and hence can be recovered by a process of 1D correlation, even over large image displacements and without the need for corner matches. Furthermore, we show that the magnitude of these shifts follows a sinusoidal form along the second (orientation) dimension of the RP image. This gives the main result that ground plane motion over RP image space forms a 3D sinusoidal manifold. Simultaneous ground plane pixel grouping and recovery of the ground plane motion thus amounts to finding the FOE and then robustly fitting a 3D sinusoid to shifts of maximum correlation in RP space. The phase of the recovered sinusoid corresponds to the orientation of the vanishing line of the ground plane and the amplitude is related to the magnitude of the robot/camera translation. Recovered FOE, vanishing line and sinusoid amplitude fully define the ground plane motion (homography) across a pair of images and thus obstacles and ground plane can be segmented without any explicit knowledge of either camera parameters or camera motion.     

超高速水下航行器纵平面姿态控制研究

超高速水下航行器在水下航行时,由于大部分表面被超空泡所包覆,其纵平面的运动特性发生显著变化,使得控制难度加大.提出把空化器用作控制面,通过操纵其舵角改变量来控制航行器在纵平面的运动,从而达到间接控制航深的目的.最后运用PID算法进行仿真分析,结果表明采用此控制方法所产生的深度负偏差较小,简单易实现、且鲁捧性好.还对航行器直接深度控制进行探讨,考虑到超空泡工况下深度测量的难度较大,姿态控制具有更强的适用性.     

Robust real-time ground plane motion compensation from a moving vehicle

One method to detect obstacles from a vehicle moving on a planar road surface is the analysis of motion-compensated difference images. In this contribution, a motion compensation algorithm is presented, which computes the required image-warping parameters from an estimate of the relative motion between camera and ground plane. The proposed algorithm estimates the warping parameters from displacements at image corners and image edges. It exploits the estimated confidence of the displacements to cope robustly with outliers. Knowledge about camera calibration, measuremts from odometry, and the previous estimate are used for motion prediction and to stabilize the estimation process when there is not enough information available in the measured image displacements. The motion compensation algorithm has been integrated with modules for obstacle detection and lane tracking. This system has been integrated in experimental vehicles and runs in real time with an overall cycle of 12.5 Hz on low-cost standard hardware. Received: 23 April 1998 / Accepted: 25 August 1999     

Parametric Model of the Perspective Projection of a Road with Applications to Lane Keeping and 3D Road Reconstruction

This paper describes an algorithm for the computation of the 3D structure of a road and of the motion state of an in-vehicle CCD camera from the lane borders on the image plane. A parametric model of the projective projection of the lane borders on the image plane is introduced and a fast and reliable algorithm to compute its parameters is described. The physical significance of the model is demonstrated by showing that the model parameters completely determine the position of the vehicle inside the lane, its heading direction, and the local structure of the road. The conditions of applicability of the model are also given and the results of its application to a sequence of images taken during a test run discussed. The algorithm is suited to real time applications and indeed an implementation on the dedicated hardware of the mobile laboratory MOBLAB runs at a frame rate of 12 images per second.     

Autonomous mobile robot model predictive control

This paper presents model predictive control of an autonomous vehicle. Simulation and experimental results have been shown and compared with input–output linearization method. The results obtained show that the MPC is an efficient method that allows for accurate control and navigation of an autonomous vehicle. Model based predictive control is tested in simulations for motion on an inclined plane. This is done to prepare future work regarding the avoidance of the violation of the smoothness condition for exact linearization, while at the same time by modifying the input commands the geometric path planning results are conserved. The approach is presented for the wheel-ground slippage and tip-over avoidance of the three-wheeled vehicle for inclined plane motion. A complete three-dimensional dynamic model using Newtonian dynamics is also presented. Simulation results using a three-wheeled vehicle built in our laboratory illustrate the performance of the proposed controller.     

机动超空泡水下航行器纵平面运动特性分析

超空泡水下航行器机动运动由于有空泡的时间延迟效应和重力效应的存在,使得航行器与直航超空泡航行器的运动特性有很大的不同;进行纵平面运动分析,是对航行器实施深度控制和姿态控制的前提;考虑时间延迟效应和重力效应引起的空泡轴变形,建立超空泡航行器机动航行纵平面运动模型,推导纵平面运动传递函数,并以某小口径(约200mm)超空泡水下航行器的特征参数为例分析其运动特性;纵平面运动特性分析为超空泡水下航行器机动运动的系统设计和控制提供了重要参考。     

Low computational-complexity algorithms for vision-aided inertial navigation of micro aerial vehicles

This paper presents low computational-complexity methods for micro-aerial-vehicle localization in GPS-denied environments. All the presented algorithms rely only on the data provided by a single onboard camera and an Inertial Measurement Unit (IMU). This paper deals with outlier rejection and relative-pose estimation. Regarding outlier rejection, we describe two methods. The former only requires the observation of a single feature in the scene and the knowledge of the angular rates from an IMU, under the assumption that the local camera motion lies in a plane perpendicular to the gravity vector. The latter requires the observation of at least two features, but it relaxes the hypothesis on the vehicle motion, being therefore suitable to tackle the outlier detection problem in the case of a 6DoF motion. We show also that if the camera is rigidly attached to the vehicle, motion priors from the IMU can be exploited to discard wrong estimations in the framework of a 2-point-RANSAC-based approach. Thanks to their inherent efficiency, the proposed methods are very suitable for resource-constrained systems. Regarding the pose estimation problem, we introduce a simple algorithm that computes the vehicle pose from the observation of three point features in a single camera image, once that the roll and pitch angles are estimated from IMU measurements. The proposed algorithm is based on the minimization of a cost function. The proposed method is very simple in terms of computational cost and, therefore, very suitable for real-time implementation. All the proposed methods are evaluated on both synthetic and real data.     

Computation of discontinuous optical flow by domain decomposition and shape optimization

A well-known method for the reconstruction of motion fields from noisy image data is to determine flow fields by the minimization of a quadratic functional. The first approach of this class has been proposed by Horn and Schunck (1981). A drawback of such approaches is that an explicit representation of the discontinuities of the motion field is lacking and that, in general, the resulting flow fields approximate the motion fields only badly at the corresponding locations in the image plane. In this article, we discuss the possibility to improve the results by hypothesizing the qualitative structure of the motion field in terms of certain parameters. We decompose the image plane into disjoint sets, restrict the domain of definition of the functionals to these sets, and use the hypotheses to deform and to move the boundaries of the sets within the image plane. We discuss the range of applicability of this new technique and illustrate the algorithm by numerical examples. This article is a revised and extended version of Schnörr (1990).This work has been supported by the PROMETHEUS programme while the author was with the Fraunhofer-Institut (IITB) at Karlsruhe.     

一种基于图像的大型车辆车身长度自动测量方法

针对传统汽车尺寸测量技术在实际应用环境限制下的不足,提出一种基于图像的大型车辆车身长度测量方法。利用图像分割技术和基于图像边缘特征的匹配方法完成车身侧面全景图像拼接,并利用双目测距原理计算出车辆与相机间的距离,在此基础上,根据透视投影原理计算出最终的车身长度。实验结果表明,该方法可以很好地在受限的测量环境中自动、快速、准确地测量出大型车辆车身长度。     

The use of optical flow for the analysis of non-rigid motions

This paper analysis the 2D motion field on the image plane produced by the 3D motion of a plane undergoing simple deformations. When the deformation can be represented by a planar linear vector field, the projected vector field, i.e., the 2D motion field of the deformation, is at most quadratic. This 2D motion field has one singular point, with eigenvalues identical to those of the singular point describing the deformation. As a consequence, the nature of the singular point of the deformation is a projective invariant. When the plane moves and experiences a linear deformation at the same time, the associated 2D motion field is at most quadratic with at most 3 singular points. In the case of a normal rototranslation, i.e., when the angular velocity is normal to the plane, and of a linear deformation, the 2D motion field has one singular point and substantial information on the rigid motion and on the deformation can be recovered from it. Experiments with image sequences of planes moving and undergoing linear deformations show that the proposed analysis can provide accurate results. In addition, experiments with deformable objects, such as water, oil, textiles and rubber show that the proposed approach can provide information on more general 3D deformations.     

智能交通卡口监测系统的图像序列压缩编码

根据智能交通卡口监测系统网像序列的特点,提出一种基于车辆目标检测与运动估计的灰度图像序列压缩编码方案,该方案分为帧内编码和帧问编码2个部分.帧内编码,即静止图像编码,采用多级树集合分裂算法对图像的提升9/7小波变换系数进行编码.帧间编码则提取图像序列中的车辆目标,应用双线性插值方法,根据前一帧中的匹配目标预测当前帧中的...     

Autonomous longitudinal motion of a paraglider. Mathematical simulation, synthesis of control

Mathematical model of motion of a paraglider in the longitudinal plane is constructed. The vehicle consists of a sail and a gondola. Both bodies are assumed to be perfectly rigid. They are connected by slings which are assumed to be perfectly rigid rods. Thus, the considered model of the paraglider represents one rigid body with three degrees of freedom. An engine, which develops thrust using a propeller, is mounted rigidly on the gondola of the vehicle. The orientation of the thrust vector with respect to the gondola is constant. The steady-state regimes of motion of the paraglider for the constant thrust are found. The law of automatic thrust control for which the flight of the vehicle is stabilized at the given altitude is designed. The domains of asymptotic stability of the paraglider motion at a constant altitude, including with account of delay, are constructed in the plane of the feedback coefficients. In this plane the domains in which a given stability factor is ensured are constructed. Some results of numerical simulation of the flight of the vehicle are presented.     

Interpretation of motion trajectories using focus of expansion

The focus of expansion (FOE) of a group of motion trajectories is defined to be a point in the image plane at which the trajectories intersect when they are extended. The FOE observed over a time sequence defines the locus of the FOE. The authors present an analytical approach for the study of dynamic events as they project on the image plane by analyzing the locus of the FOE. They find that the locus of the FOE can be used to make qualitative assertions regarding the type of motion. Interesting behavior of the locus of the FOE for various types of motion is observed. The cases include a single point and a horizontal, a vertical, and a sloped straight line. It was also possible to determine whether the object has approaching or receding motion or when the object changes its direction of motion. This inference can be used in qualitative computer vision     

Quantitative planar region detection

This paper presents a means of segmenting planar regions from two views of a scene using point correspondences. The initial selection of groups of coplanar points is performed on the basis of conservation of two five point projective invariants (groups for which this invariant is conserved are assumed to be coplanar). The five point correspondences are used to estimate a projectivity which is used to predict the change in position of other points assuming they lie on the same plane as the original four. The variance in any points new position is then used to define a distance threshold between actual and predicted position which is used as a coplanarity test to find extended planar regions. If two distinct planar regions can be found then a novel motion direction estimator suggests itself. The projection of the line of intersection of two planes in an image may also be recovered. An analytical error model is derived which relates image uncertainty in a corner's position to genuine perpendicular height of a point above a given plane in the world. The model may be used for example to predict the performance of given stereo ground plane prediction system or a monocular drivable region detection system on and AGV. The model may also be used in reverse to determine the camera resolution required if a vehicle in motion is to resolve obstacles of a given height a given distance from it.     

基于平面拟合恢复运动矢量的错误隐藏算法

针对压缩视频码流在无线信道传输过程中由于运动矢量失配导致重构图像质量的问题,提出了一种基于平面拟合恢复运动矢量的错误隐藏算法。首先,根据H.264视频标准具有多参考帧且相邻宏块间的相关性的特点,通过把与受损宏块直接相邻的每个分块的运动矢量定义为一个点,采用平面拟合方法表征小范围内相邻运动矢量的变化趋势,对受损宏块运动矢量进行重建,然后利用改进的边界匹配函数,选取最优运动矢量对受损图像进行恢复。实验结果表明,该算法不仅避免了AIC算法产生的块效应,而且在不同的RTP丢包概率下,该算法比AIC算法得到的峰值信噪比有0.3~2.5 dB的提升。     

Low-cost sensor to detect overtaking based on optical flow

The automotive industry invests substantial amounts of money in driver-security and driver-assistance systems. We propose an overtaking detection system based on visual motion cues that combines feature extraction, optical flow, solid-objects segmentation and geometry filtering, working with a low-cost compact architecture based on one focal plane and an on-chip embedded processor. The processing is divided into two stages: firstly analog processing on the focal plane processor dedicated to image conditioning and relevant image-structure selection, and secondly, vehicle tracking and warning-signal generation by optical flow, using a simple digital microcontroller. Our model can detect an approaching vehicle (multiple-lane overtaking scenarios) and warn the driver about the risk of changing lanes. Thanks to the use of tightly coupled analog and digital processors, the system is able to perform this complex task in real time with very constrained computing resources. The proposed method has been validated with a sequence of more than 15,000 frames (90 overtaking maneuvers) and is effective under different traffic situations, as well as weather and illumination conditions.