How Optimal Depth Cue Integration Depends on the Task

The aim of this paper is to explore a linear geometric algorithm for recovering the three dimensional motion of a moving camera from image velocities. Generic similarities and differences between the discrete approach and the differential approach are clearly revealed through a parallel development of an analogous motion estimation theory previously explored in Vieville, T. and Faugeras, O.D. 1995. In Proceedings of Fifth International Conference on Computer Vision, pp. 750–756; Zhuang, X. and Haralick, R.M. 1984. In Proceedings of the First International Conference on Artificial Intelligence Applications, pp. 366–375. We present a precise characterization of the space of differential essential matrices, which gives rise to a novel eigenvalue-decomposition-based 3D velocity estimation algorithm from the optical flow measurements. This algorithm gives a unique solution to the motion estimation problem and serves as a differential counterpart of the well-known SVD-based 3D displacement estimation algorithm for the discrete case. Since the proposed algorithm only involves linear algebra techniques, it may be used to provide a fast initial guess for more sophisticated nonlinear algorithms (Ma et al., 1998c. Electronic Research Laboratory Memorandum, UC Berkeley, UCB/ERL(M98/37)). Extensive simulation results are presented for evaluating the performance of our algorithm in terms of bias and sensitivity of the estimates with respect to different noise levels in image velocity measurements.     

A hierarchy of cameras for 3D photography

The view-independent visualization of 3D scenes is most often based on rendering accurate 3D models or utilizes image-based rendering techniques. To compute the 3D structure of a scene from a moving vision sensor or to use image-based rendering approaches, we need to be able to estimate the motion of the sensor from the recorded image information with high accuracy, a problem that has been well-studied. In this work, we investigate the relationship between camera design and our ability to perform accurate 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for which the motion estimation problem can be solved independently of the depth of the scene which leads to fast and robust algorithms for 3D Photography. In between are multiple view cameras with a large field of view which we have built, as well as omni-directional sensors.     

LANDING AN UNMANNED AIR VEHICLE: VISION BASED MOTION ESTIMATION AND NONLINEAR CONTROL

In this paper, we use computer vision as a feedback sensor in a control loop for landing an unmanned air vehicle (UAV) on a landing pad. The vision problem we address here is then a special case of the classic ego-motion estimation problem since all feature points lie on a planar surface (the landing pad). We study together the discrete and differential versions of the ego-motion estimation, in order to obtain both position and velocity of the UAV relative to the landing pad. After briefly reviewing existing algorithm for the discrete case, we present, in a unified geometric framework, a new estimation scheme for solving the differential case. We further show how the obtained algorithms enable the vision sensor to be placed in the feedback loop as a state observer for landing control. These algorithms are linear, numerically robust, and computationally inexpensive hence suitable for real-time implementation. We present a thorough performance evaluation of the motion estimation algorithms under varying levels of image measurement noise, altitudes of the camera above the landing pad, and different camera motions relative to the landing pad. A landing controller is then designed for a full dynamic model of the UAV. Using geometric nonlinear control theory, the dynamics of the UAV are decoupled into an inner system and outer system. The proposed control scheme is then based on the differential flatness of the outer system. For the overall closed-loop system, conditions are provided under which exponential stability can be guaranteed. In the closed-loop system, the controller is tightly coupled with the vision based state estimation and the only auxiliary sensor are accelerometers for measuring acceleration of the UAV. Finally, we show through simulation results that the designed vision-in-the-loop controller generates stable landing maneuvers even for large levels of image measurement noise. Experiments on a real UAV will be presented in future work.     

A structured light vision system for out-of-plane vibration frequencies location of a moving web

In this paper, we address the problem of the detection of out-of-plane web vibrations by means of a single camera and a laser dots pattern device. We have been motivated by the important economical impact of web vibrations phenomena which occur in winding/unwinding systems. Among many sources of disturbances, out-of-plane vibrations of an elastic moving web are well-known to be one of the most limiting factors for the velocity in the web transport industry.The new technique we proposed for the contact-less estimation of out-of-plane web vibration properties and during the winding process is the main contribution of this work. As far as we know, this is the first time a technique is proposed to evaluate the vibrations of a moving web with a camera. Vibration frequencies are estimated from distance variations of a web cross-section with respect to the camera.Experiments have been performed on a winding plant for elastic fabric with a web width of 10 cm. Distances from the web surface to the camera have been estimated all along an image sequence and the most significant frequencies have been extracted from the variations of this signal (forced and free vibrations) and compared to those provided with strain gauges and also with a simple elastic string model, in motion.     

Distance estimation using inertial sensor and vision

A simple distance estimation algorithm using inertial sensors and a mono camera is proposed. Two images of a target are obtained by moving a mono camera. The movement of the camera is estimated using inertial sensors and used as the baseline for the distance estimation. Through experiments, the accuracy of the proposed method is evaluated, where the distance error is 3.9% on average in a few meter ranges.     

分裂合并运动分割的多运动视觉里程计方法

目的 视觉里程计（visual odometry,VO）仅需要普通相机即可实现精度可观的自主定位,已经成为计算机视觉和机器人领域的研究热点,但是当前研究及应用大多基于场景为静态的假设,即场景中只有相机运动这一个运动模型,无法处理多个运动模型,因此本文提出一种基于分裂合并运动分割的多运动视觉里程计方法,获得场景中除相机运动外多个运动目标的运动状态。方法 基于传统的视觉里程计框架,引入多模型拟合的方法分割出动态场景中的多个运动模型,采用RANSAC（random sample consensus）方法估计出多个运动模型的运动参数实例;接着将相机运动信息以及各个运动目标的运动信息转换到统一的坐标系中,获得相机的视觉里程计结果,以及场景中各个运动目标对应各个时刻的位姿信息;最后采用局部窗口光束法平差直接对相机的姿态以及计算出来的相机相对于各个运动目标的姿态进行校正,利用相机运动模型的内点和各个时刻获得的相机相对于运动目标的运动参数,对多个运动模型的轨迹进行优化。结果 本文所构建的连续帧运动分割方法能够达到较好的分割结果,具有较好的鲁棒性,连续帧的分割精度均能达到近100%,充分保证后续估计各个运动模型参数的准确性。本文方法不仅能够有效估计出相机的位姿,还能估计出场景中存在的显著移动目标的位姿,在各个分段路径中相机自定位与移动目标的定位结果位置平均误差均小于6%。结论 本文方法能够同时分割出动态场景中的相机自身运动模型和不同运动的动态物体运动模型,进而同时估计出相机和各个动态物体的绝对运动轨迹,构建出多运动视觉里程计过程。     

Visually guided ranging from observations of points, lines and curves via an identifier based nonlinear observer

In this paper we consider the problem of estimating the range information of features on an affine plane in by observing its image with the aid of a CCD camera, wherein we assume that the camera is undergoing a known motion. The features considered are points, lines and planar curves located on planar surfaces of static objects. The dynamics of the moving projections of the features on the image plane have been described as a suitable differential equation on an appropriate feature space. This dynamics is used to estimate feature parameters from which the range information is readily available. In this paper the proposed identification has been carried out via a newly introduced identifier based observer. Performance of the observer has been studied via simulation.     

Vehicle collision risk estimation based on RGB-D camera for urban road

Traffic violation is the main cause of traffic accidents. To reduce the incidence of traffic accidents, the common practice at present is to strength the penalties for traffic violation. However, little attention has been paid to issue warning for dangerous driving behaviors, especially for the case where two vehicles have a good chance of collision. In this paper, a framework for collision risk estimation using RGB-D camera is proposed for vehicles running on the urban road, where the depth information is fused with the video information for accurate calculation of the position and speed of the vehicles, two essential parameters for motion trajectory estimation. Considering that the motion trajectory or its differences can be considered as a steady signal, a method based on autoregressive integrated moving average (ARIMA) models is presented to predict vehicle trajectory. Then, the collision risk is estimated based on the predicted trajectory. The experiments are carried out on the data from the real vehicles. The result shows that the accuracy of position and speed estimation can be guaranteed within urban road and the error of trajectory prediction is very minor which is unlikely to have a significant impact on calculating the probability of collision in most situations, so the proposed framework is effective in collision risk estimation.     

Direct visual tracking control of remote cellular robots

This paper presents the design of a stable non-linear control system for the remote visual tracking of cellular robots. The robots are controlled through visual feedback based on the processing of the image captured by a fixed video camera observing the workspace. The control algorithm is based only on measurements on the image plane of the visual camera–direct visual control–thus avoiding the problems related to camera calibration. In addition, the camera plane may have any (unknown) orientation with respect to the robot workspace. The controller uses an on-line estimation of the image Jacobians. Considering the Jacobians’ estimation errors, the control system is capable of tracking a reference point moving on the image plane–defining the reference trajectory–with an ultimately bounded error. An obstacle avoidance strategy is also developed in the same context, based on the visual impedance concept. Experimental results show the performance of the overall control system.     

基于H.264压缩域的镜头运动信息提取

本文根据H．264编码特点，提出了一种从H.264压缩域中快速有效地进行镜头运动参数估计的算法。镜头运动信息在视频分析处理中起着重要的作用。该算法充分利用了H．264压缩码流中的信息，通过提取预测残差DC图像的运动背景区域，估计镜头运动参数，从而保证了参数估计的准确性，有效地克服了已有文献中仅仅采用运动矢量进行镜头运动估计的局限性。根据不同的H．264测试序列的对比分析，结果表明本算法可快速准确地对H．264视频序列进行全局运动信息估计，同时具有很高的鲁棒性。     

Multiple motion scene reconstruction with uncalibrated cameras

In this paper, we describe a reconstruction method for multiple motion scenes, which are scenes containing multiple moving objects, from uncalibrated views. Assuming that the objects are moving with constant velocities, the method recovers the scene structure, the trajectories of the moving objects, the camera motion, and the camera intrinsic parameters (except skews) simultaneously. We focus on the case where the cameras have unknown and varying focal lengths while the other intrinsic parameters are known. The number of the moving objects is automatically detected without prior motion segmentation. The method is based on a unified geometrical representation of the static scene and the moving objects. It first performs a projective reconstruction using a bilinear factorization algorithm and, then, converts the projective solution to a Euclidean one by enforcing metric constraints. Experimental results on synthetic and real images are presented.     

基于分布估计的离散差分演化算法

差分演化(DE)是解决优化问题的非常有效的新兴智能算法,但它主要用于连续优化领域,至今尚不能象解决连续优化问题那样有效的处理组合优化问题.首先提出了离散DE用于组合优化问题,然后在离散DE中引入分布估计算法(EDA)来提高性能,把EDA抽样得到的全局统计信息和离散DE获得的局部演化信息相结合来产生新解,形成基于EDA的离散DE算法.为了保持种群多样性,在提出的算法中引入了位翻转变异操作.实验结果表明,EDA能大大提高离散DE的性能.     

Detection of moving foreground objects in videos with strong camera motion

In this paper, we propose a novel method for moving foreground object extraction in sequences taken by a wearable camera, with strong motion. We use camera motion compensated frame differencing, enhanced with a novel kernel-based estimation of the probability density function of background pixels. The probability density functions are used for filtering false foreground pixels on the motion compensated difference frame. The estimation is based on a limited number of measurements; therefore, we introduce a special, spatio-temporal sample point selection and an adaptive thresholding method to deal with this challenge. Foreground objects are built with the DBSCAN algorithm from detected foreground pixels.     

Fast image motion segmentation for surveillance applications

Wireless, battery-powered camera networks are becoming of increasing interest for surveillance and monitoring applications. The computational power of these platforms is often limited in order to reduce energy consumption. In addition, many embedded processors do not have floating point support in hardware. Among the visual tasks that a visual sensor node may be required to perform, motion analysis is one of the most basic and relevant. Events of interest are usually characterized by the presence of moving objects or persons. Knowledge of the direction of motion and velocity of a moving body may be used to take actions such as sending an alarm or triggering other camera nodes in the network.We present a fast algorithm for identifying moving areas in an image. The algorithm is efficient and amenable to implementation in fixed point arithmetic. Once the moving blobs in an image have been precisely localized, the average velocity vector can be computed using a small number of floating point operations. Our procedure starts by determining an initial labeling of image blocks based on local differential analysis. Then, belief propagation is used to impose spatial coherence and to resolve aperture effect inherent in texture less areas. A detailed analysis of the computational cost of the algorithm and of the provisions that must be taken in order to avoid overflow with 32-bit words is included.     

具有深度自适应估计的视觉伺服优化

在手眼机器人视觉伺服中,如何确定机器人末端摄像机移动的速度和对物体的深度进行有效的估计还没有较好的解决方法.本文采用一般模型法,通过求解最优化控制问题来设计摄像机的速度,同时,利用物体初始及期望位置的深度估计值,提出了一种自适应估计的算法对物体的深度进行估计,给出了深度变化趋势,实现了基于图像的定位控制.该方法能够使机器人在工作空间范围内从任一初始位置出发到达期望位置,实现了系统的全局渐近稳定且不需要物体的几何模型及深度的精确值.最后给出的仿真实例表明了本方法的有效性.     

Fast Electronic Digital Image Stabilization for Off-Road Navigation

Image stabilization can be used as front-end system for many tasks that require dynamic image analysis, such as navigation and tracking of independently moving objects from a moving platform. We present a fast and robust electronic digital image stabilization system that can handle large image displacements based on a two-dimensional feature-based multi-resolution motion estimation technique. The method tracks a small set of features and estimates the movement of the camera between consecutive frames. Stabilization is achieved by combining all motion from a reference frame and warping the current frame back to the reference. The system has been implemented on parallel pipeline image processing hardware (a Datacube MaxVideo 200) connected to a SUN SPARCstation 20/612 via a VME bus adaptor. Experimental results using video sequences taken from a camera mounted on a vehicle moving on rough terrain show the robustness of the system while running at approximately 20 frames/s.     

Parametric ego-motion estimation for vehicle surround analysis using an omnidirectional camera

Omnidirectional cameras that give a 360° panoramic view of the surroundings have recently been used in many applications such as robotics, navigation, and surveillance. This paper describes the application of parametric ego-motion estimation for vehicle detection to perform surround analysis using an automobile-mounted camera. For this purpose, the parametric planar motion model is integrated with the transformations to compensate distortion in omnidirectional images. The framework is used to detect objects with independent motion or height above the road. Camera calibration as well as the approximate vehicle speed obtained from a CAN bus are integrated with the motion information from spatial and temporal gradients using a Bayesian approach. The approach is tested for various configurations of an automobile-mounted omni camera as well as a rectilinear camera. Successful detection and tracking of moving vehicles and generation of a surround map are demonstrated for application to intelligent driver support.Received: 1 August 2003, Accepted: 8 July 2004, Published online: 3 February 2005     

An effective algorithm to detect both smoke and flame using color and wavelet analysis

Fire detection is an important task in many applications. Smoke and flame are two essential symbols of fire in images. In this paper, we propose an algorithm to detect smoke and flame simultaneously for color dynamic video sequences obtained from a stationary camera in open space. Motion is a common feature of smoke and flame and usually has been used at the beginning for extraction from a current frame of candidate areas. The adaptive background subtraction has been utilized at a stage of moving detection. In addition, the optical flow-based movement estimation has been applied to identify a chaotic motion. With the spatial and temporal wavelet analysis, Weber contrast analysis and color segmentation, we achieved moving blobs classification. Real video surveillance sequences from publicly available datasets have been used for smoke detection with the utilization of our algorithm. We also have conducted a set of experiments. Experiments results have shown that our algorithm can achieve higher detection rate of 87% for smoke and 92% for flame.     

基于深度学习的单目深度估计技术综述

场景的深度估计问题是计算机视觉领域中的经典问题之一,也是3维重建和图像合成等应用中的一个重要环节。基于深度学习的单目深度估计技术高速发展,各种网络结构相继提出。本文对基于深度学习的单目深度估计技术最新进展进行了综述,回顾了基于监督学习和基于无监督学习方法的发展历程。重点关注单目深度估计的优化思路及其在深度学习网络结构中的表现,将监督学习方法分为多尺度特征融合的方法、结合条件随机场(conditional random field,CRF)的方法、基于序数关系的方法、结合多元图像信息的方法和其他方法等5类;将无监督学习方法分为基于立体视觉的方法、基于运动恢复结构(structure from motion,SfM)的方法、结合对抗性网络的方法、基于序数关系的方法和结合不确定性的方法等5类。此外,还介绍了单目深度估计任务中常用的数据集和评价指标,并对目前基于深度学习的单目深度估计技术在精确度、泛化性、应用场景和无监督网络中不确定性研究等方面的现状和面临的挑战进行了讨论,为相关领域的研究人员提供一个比较全面的参考。     

Moving object detection and segmentation in urban environments from a moving platform

This paper proposes an effective approach to detect and segment moving objects from two time-consecutive stereo frames, which leverages the uncertainties in camera motion estimation and in disparity computation. First, the relative camera motion and its uncertainty are computed by tracking and matching sparse features in four images. Then, the motion likelihood at each pixel is estimated by taking into account the ego-motion uncertainty and disparity in computation procedure. Finally, the motion likelihood, color and depth cues are combined in the graph-cut framework for moving object segmentation. The efficiency of the proposed method is evaluated on the KITTI benchmarking datasets, and our experiments show that the proposed approach is robust against both global (camera motion) and local (optical flow) noise. Moreover, the approach is dense as it applies to all pixels in an image, and even partially occluded moving objects can be detected successfully. Without dedicated tracking strategy, our approach achieves high recall and comparable precision on the KITTI benchmarking sequences.