Accurate optical flow computation under non-uniform brightness variations

In this paper, we present a very accurate algorithm for computing optical flow with non-uniform brightness variations. The proposed algorithm is based on a generalized dynamic image model (GDIM) in conjunction with a regularization framework to cope with the problem of non-uniform brightness variations. To alleviate flow constraint errors due to image aliasing and noise, we employ a reweighted least-squares method to suppress unreliable flow constraints, thus leading to robust estimation of optical flow. In addition, a dynamic smoothness adjustment scheme is proposed to efficiently suppress the smoothness constraint in the vicinity of the motion and brightness variation discontinuities, thereby preserving motion boundaries. We also employ a constraint refinement scheme, which aims at reducing the approximation errors in the first-order differential flow equation, to refine the optical flow estimation especially for large image motions. To efficiently minimize the resulting energy function for optical flow computation, we utilize an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the large linear system. Experimental results on some synthetic and real image sequences show that the proposed algorithm compares favorably to most existing techniques reported in literature in terms of accuracy in optical flow computation with 100% density.     

Monocular scene flow estimation via variational method

Scene flow provides the 3D motion field of point clouds, which correspond to image pixels. Current algorithms usually need complex stereo calibration before estimating flow, which has strong restrictions on the position of the camera. This paper proposes a monocular camera scene flow estimation algorithm. Firstly, an energy functional is constructed, where three important assumptions are turned into data terms derivation: a brightness constancy assumption, a gradient constancy assumption, and a short time object velocity constancy assumption. Two smooth operators are used as regularization terms. Then, an occluded map computation algorithm is used to ensure estimating scene flow only on un-occluded points. After that, the energy functional is solved with a coarse-to-fine variational equation on Gaussian pyramid, which can prevent the iteration from converging to a local minimum value. The experiment results show that the algorithm can use three sequential frames at least to get scene flow in world coordinate, without optical flow or disparity inputting.     

Dynamic Texture Detection Based on Motion Analysis

Motion estimation is usually based on the brightness constancy assumption. This assumption holds well for rigid objects with a Lambertian surface, but it is less appropriate for fluid and gaseous materials. For these materials an alternative assumption is required. This work examines three possible alternatives: gradient constancy, color constancy and brightness conservation (under this assumption the brightness of an object can diffuse to its neighborhood). Brightness conservation and color constancy are found to be adequate models. We propose a method for detecting regions of dynamic texture in image sequences. Accurate segmentation into regions of static and dynamic texture is achieved using a level set scheme. The level set function separates each image into regions that obey brightness constancy and regions that obey the alternative assumption. We show that the method can be simplified to obtain a less robust but fast algorithm, capable of real-time performance. Experimental results demonstrate accurate segmentation by the full level set scheme, as well as by the simplified method. The experiments included challenging image sequences, in which color or geometry cues by themselves would be insufficient.     

Robust Optical Flow Computation Based on Least-Median-of-Squares Regression

An optical flow estimation technique is presented which is based on the least-median-of-squares (LMedS) robust regression algorithm enabling more accurate flow estimates to be computed in the vicinity of motion discontinuities. The flow is computed in a blockwise fashion using an affine model. Through the use of overlapping blocks coupled with a block shifting strategy, redundancy is introduced into the computation of the flow. This eliminates blocking effects common in most other techniques based on blockwise processing and also allows flow to be accurately computed in regions containing three distinct motions.A multiresolution version of the technique is also presented, again based on LMedS regression, which enables image sequences containing large motions to be effectively handled.An extensive set of quantitative comparisons with a wide range of previously published methods are carried out using synthetic, realistic (computer generated images of natural scenes with known flow) and natural images. Both angular and absolute flow errors are calculated for those sequences with known optical flow. Displaced frame difference error, used extensively in video compression, is used for those natural scenes with unknown flow. In all of the sequences tested, a comparison with those methods that result in a dense flow field (greater than 80% spatial coverage), show that the LMedS technique produces the least error irrespective of the error measure used.     

A Variational Approach to Video Registration with Subspace Constraints

This paper addresses the problem of non-rigid video registration, or the computation of optical flow from a reference frame to each of the subsequent images in a sequence, when the camera views deformable objects. We exploit the high correlation between 2D trajectories of different points on the same non-rigid surface by assuming that the displacement of any point throughout the sequence can be expressed in a compact way as a linear combination of a low-rank motion basis. This subspace constraint effectively acts as a trajectory regularization term leading to temporally consistent optical flow. We formulate it as a robust soft constraint within a variational framework by penalizing flow fields that lie outside the low-rank manifold. The resulting energy functional can be decoupled into the optimization of the brightness constancy and spatial regularization terms, leading to an efficient optimization scheme. Additionally, we propose a novel optimization scheme for the case of vector valued images, based on the dualization of the data term. This allows us to extend our approach to deal with colour images which results in significant improvements on the registration results. Finally, we provide a new benchmark dataset, based on motion capture data of a flag waving in the wind, with dense ground truth optical flow for evaluation of multi-frame optical flow algorithms for non-rigid surfaces. Our experiments show that our proposed approach outperforms state of the art optical flow and dense non-rigid registration algorithms.     

一种基于线性亮度变化模型的鲁棒的光流算法

提出了一种鲁棒光流算法,用于计算光照强度、帧间运动速度及运动速度变化较大情况下的光流场。在梯度约束方程中嵌入了线性亮度变化模型,以提高大的光照强度变化下算法稳健性;将各向异性扩散方程引入空间方向平滑约束,以改善运动不连续处的流速计算精度,并依此建立了多尺度空间微分光流算法。参数的均衡化得到了线性尺度变化下的恒定能量函数。迭代运算引入运动补偿的概念,使亮度误差减小。实验结果表明,在光照强度和运动速度及速度变化较大时,本文算法具有很好的计算精度,并产生密度100%的光流场。     

Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy

Single-scale approaches to the determination of the optical flow field from the time-varying brightness pattern assume that spatio-temporal discretization is adequate for representing the patterns and motions in a scene. However, the choice of an appropriate spatial resolution is subject to conflicting, scene-dependent, constraints. In intensity-base methods for recovering optical flow, derivative estimation is more accurate for long wavelengths and slow velocities (with respect to the spatial and temporal discretization steps). On the contrary, short wavelengths and fast motions are required in order to reduce the errors caused by noise in the image acquisition and quantization process.Estimating motion across different spatial scales should ameliorate this problem. However, homogeneous multiscale approaches, such as the standard multigrid algorithm, do not improve this situation, because an optimal velocity estimate at a given spatial scale is likely to be corrupted at a finer scale. We propose an adaptive multiscale method, where the discretization scale is chosen locally according to an estimate of the relative error in the velocity estimation, based on image properties.Results for synthetic and video-acquired images show that our coarse-to-fine method, fully parallel at each scale, provides substantially better estimates of optical flow than do conventional algorithms, while adding little computational cost.     

一种改进的变分光流计算方法

提出一种改进的变分光流算法的能量泛函,该能量泛函的数据项由灰度不变假设和Hessian矩阵不变假设组成,并与Lucas局部光流一致方法相结合。平滑项的设计采用先各项同性平滑再各项异性平滑的策略,其中引入图像一致增强思想。实验结果证明,运用该方法进行光流计算的效果比以往变分方法有所改进。     

Multi-view Scene Flow Estimation: A View Centered Variational Approach

We present a novel method for recovering the 3D structure and scene flow from calibrated multi-view sequences. We propose a 3D point cloud parametrization of the 3D structure and scene flow that allows us to directly estimate the desired unknowns. A unified global energy functional is proposed to incorporate the information from the available sequences and simultaneously recover both depth and scene flow. The functional enforces multi-view geometric consistency and imposes brightness constancy and piecewise smoothness assumptions directly on the 3D unknowns. It inherently handles the challenges of discontinuities, occlusions, and large displacements. The main contribution of this work is the fusion of a 3D representation and an advanced variational framework that directly uses the available multi-view information. This formulation allows us to advantageously bind the 3D unknowns in time and space. Different from optical flow and disparity, the proposed method results in a nonlinear mapping between the images’ coordinates, thus giving rise to additional challenges in the optimization process. Our experiments on real and synthetic data demonstrate that the proposed method successfully recovers the 3D structure and scene flow despite the complicated nonconvex optimization problem.     

Computing optical flow with physical models of brightness variation

Although most optical flow techniques presume brightness constancy, it is well-known that this constraint is often violated, producing poor estimates of image motion. This paper describes a generalized formulation of optical flow estimation based on models of brightness variations that are caused by time-dependent physical processes. These include changing surface orientation with respect to a directional illuminant, motion of the illuminant, and physical models of heat transport in infrared images. With these models, we simultaneously estimate the 2D image motion and the relevant physical parameters of the brightness change model. The estimation problem is formulated using total least squares, with confidence bounds on the parameters. Experiments in four domains, with both synthetic and natural inputs, show how this formulation produces superior estimates of the 2D image motion     

Optical flow estimation: an error analysis of gradient-based methods with local optimization

Multiple views of a scene can provide important information about the structure and dynamic behavior of three-dimensional objects. Many of the methods that recover this information require the determination of optical flow-the velocity, on the image, of visible points on object surfaces. An important class of techniques for estimating optical flow depend on the relationship between the gradients of image brightness. While gradient-based methods have been widely studied, little attention has been paid to accuracy and reliability of the approach. Gradient-based methods are sensitive to conditions commonly encountered in real imagery. Highly textured surfaces, large areas of constant brightness, motion boundaries, and depth discontinuities can all be troublesome for gradient-based methods. Fortunately, these problematic areas are usually localized can be identified in the image. In this paper we examine the sources of errors for gradient-based techniques that locally solve for optical flow. These methods assume that optical flow is constant in a small neighborhood. The consequence of violating in this assumption is examined. The causes of measurement errors and the determinants of the conditioning of the solution system are also considered. By understanding how errors arise, we are able to define the inherent limitations of the technique, obtain estimates of the accuracy of computed values, enhance the performance of the technique, and demonstrate the informative value of some types of error.     

A non-regression-coefficients method of sea surface temperature retrieval from space

For several years now NOAA/NESDIS have derived an operational global sea surface temperature (SST) product from the AVHRR instrument on the NOAA satellites. This is done using the MCSST and CPSST algorithms which contain coefficients that are determined from a regression analysis of satellite data against in situ surface data. The current algorithms are used to provide global SST data without taking into account the latitude, climate or location of the satellite data, although the CPSST coefficients do have a weak dependence on the satellite brightness temperatures. Because of this global application the current SST algorithms have inherent errors due to local climate influences. In this paper a new SST algorithm is developed that does not rely on regression analysis to derive its coefficients. By using the spatial variation of the brightness temperatures in a small area (50 km by 50 km) it is possible to derive the appropriate coefficients to use in the algorithm. The SST field can thus be derived at any location without need for prior determination of the algorithm coefficients. In a simulation study, data from twenty-five radiosonde ascents-arc use with an atmospheric transmission model to derive a range of atmospheric transmittances and satellite brightness temperatures. Coincident AVHRR data and ship data are used to assess the accuracy of the new algorithm. The various dependencies of the terms in the SST algorithm are investigated. As with the MCSST and CPSST algorithms, the new method has largest errors when applied in situations of abnormal atmospheric structure. The improvement over the MCSST product may initially be only marginal, but with the advent of the more precise data from the Along Track Scanning Radiometer (ATSR) a more accurate global SST product may be possible.     

干涉式被动微波成像仪二维频谱特征及误差分析

干涉式被动微波成像仪(干涉式综合孔径微波辐射计)利用不同距离的干涉天线对形成的基线对视场范围内亮温分布的空间频谱进行采样,进而反演得到亮温图像。首先介绍了干涉式被动微波成像仪的基本工作原理,在此基础上分析了空间图像的二维频谱特征,并能利用这些特征从采样频谱中初步分析出原始亮温中的某些特殊分布,证实了在反演亮温存在明显振荡的情况下,可以从振荡方向判断存在误差的基线的大致位置。还分析了成像仪的天线位置误差、信道的幅度和相位误差以及天线方向图误差对反演亮温的影响及特点。这些分析将为从反演图像判断干涉式被动微波成像仪误差类型和来源提供重要的判别依据,并为后续的反演图像增强算法的设计提供重要的参考。     

Highly Accurate Optic Flow Computation with Theoretically Justified Warping

In this paper, we suggest a variational model for optic flow computation based on non-linearised and higher order constancy assumptions. Besides the common grey value constancy assumption, also gradient constancy, as well as the constancy of the Hessian and the Laplacian are proposed. Since the model strictly refrains from a linearisation of these assumptions, it is also capable to deal with large displacements. For the minimisation of the rather complex energy functional, we present an efficient numerical scheme employing two nested fixed point iterations. Following a coarse-to-fine strategy it turns out that there is a theoretical foundation of so-called warping techniques hitherto justified only on an experimental basis. Since our algorithm consists of the integration of various concepts, ranging from different constancy assumptions to numerical implementation issues, a detailed account of the effect of each of these concepts is included in the experimental section. The superior performance of the proposed method shows up by significantly smaller estimation errors when compared to previous techniques. Further experiments also confirm excellent robustness under noise and insensitivity to parameter variations.     

Reliable and Efficient Computation of Optical Flow

In this paper, we present two very efficient and accurate algorithms for computing optical flow. The first is a modified gradient-based regularization method, and the other is an SSD-based regularization method. For the gradient-based method, to amend the errors in the discrete image flow equation caused by numerical differentiation as well as temporal and spatial aliasing in the brightness function, we propose to selectively combine the image flow constraint and a contour-based flow constraint into the data constraint by using a reliability measure. Each data constraint is appropriately normalized to obtain an approximate minimum distance (of the data point to the linear flow equation) constraint instead of the conventional linear flow constraint. These modifications lead to robust and accurate optical flow estimation. We propose an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the resulting large and sparse linear system efficiently. Our SSD-based regularization method uses a normalized SSD measure (based on a similar reasoning as in the gradient-based scheme) as the data constraint in a regularization framework. The nonlinear conjugate gradient algorithm in conjunction with an incomplete Cholesky preconditioning is developed to solve the resulting nonlinear minimization problem. Experimental results on synthetic and real image sequences for these two algorithms are given to demonstrate their performance in comparison with competing methods reported in literature.     

Variational-Bayes Optical Flow

The Horn-Schunck (HS) optical flow method is widely employed to initialize many motion estimation algorithms. In this work, a variational Bayesian approach of the HS method is presented, where the motion vectors are considered to be spatially varying Student’s t-distributed unobserved random variables, i.e., the prior is a multivariate Student’s t-distribution, while the only observations available is the temporal and spatial image difference. The proposed model takes into account the residual resulting from the linearization of the brightness constancy constraint by Taylor series approximation, which is also assumed to be a spatially varying Student’s t-distributed observation noise. To infer the model variables and parameters we recur to variational inference methodology leading to an expectation-maximization (EM) framework with update equations analogous to the Horn-Schunck approach. This is accomplished in a principled probabilistic framework where all of the model parameters are estimated automatically from the data. Experimental results show the improvement obtained by the proposed model which may substitute the standard algorithm in the initialization of more sophisticated optical flow schemes.     

Determination of Horizontal Motion through Optical Flow Computations

For intelligent/autonomous subsea vehicles,reliable short-range horizontal positioning is difficult to achieve,particularly over flat bottom topography.A potential solution proposed in this paper utilized a passive optical sensing method to estimate the vehicle displacement using the bottom surface texture.The suggested optical flow method does not require any feature correspondences in images and it is robust in allowing brightness changes between image frames.Fundamentally,this method is similar to correlation methods attempting to match images and compute the motion disparity.However,in correlation methods,searching a neighbor region blindly for best match is lengthy.Main contributions of this paper come from the analysis showing that optical flow computation based on the general model cannot avoid errors except for null motion although the sign of optical flow keeps correct,and from the development of an iterative shifting method based on the error characteristics to accurately determine motions.Advantages of the proposed method are verified by real image experiments.     

基于结构优化的DDAG-SVM上肢康复训练动作识别方法

针对上肢康复训练系统中训练评估方法核心的动作识别问题,提出一种面向Brunnstrom 4~5期患者上肢康复训练动作的SODDAG-SVM(Structure-optimized decision directed acyclic graph-support vector machine)多分类识别方法.首先将多分类问题分解成一组二分类问题,并使用支持向量机构建各二分类器,分别采用遗传算法和特征子集区分度准则对各二分类器的核函数参数及特征子集进行优化.然后使用类对的SVM二分类器泛化误差来衡量每个类对的易被分离程度,并由其建立类对泛化误差上三角矩阵.最后由根节点开始,依次根据各节点的泛化误差矩阵,通过选择其中最易被分离类对的SVM分类器构成该节点的方式,来构建SODDAG-SVM多分类器结构.当待预测的实例较少时,直接构建实例经过的SODDAG-SVM部分结构并对实例进行预测;当待预测的实例较多时,先构建完整的SODDAG-SVM结构,再代入所有实例进行预测.通过人体传感技术获得Brunnstrom 4~5阶段上肢康复训练的常用动作样本集,进行SODDAG-SVM动作识别实验,准确率达到了95.49%,结果均优于常规的决策有向无环图(Decision directed acyceic graph,DDAG)和MaxWins方法,实验表明本文方法能有效地提高上肢康复训练动作识别的准确率.     

面向光流分析的机器人红外视觉图像对预处理

为了增强机器人单目视觉系统在黑暗环境下的环境感知能力,提高其在视觉图像序列处理过程中光流场计算的准确率,对由红外摄像机所采集的红外图像序列的预处理方法进行了研究,建立了一种空域与变换域相结合的处理方法。为平衡红外图像序列对之间的亮度差异并提高图像亮度,在空域中采用直方图均衡化方法进行处理;然后,对图像进行非下采样Contourlet变换,在变换域中利用图像的强边缘、弱边缘和噪声在不同分解尺度和分解方向上具有不同几何流的性质进行区分,分别采取保留、增强以及去除处理;最后利用调整后的系数进行图像重构。该预处理方法在平衡图像亮度的同时增强了图像纹理并减少了噪声。实验结果表明,预处理程序有效地提高了红外图像对的光流有效点识别率,该方法增强了机器人单目视觉系统在黑暗环境中对环境的感知能力。     

基于不变矩景象匹配算法

分析基于灰度匹配方法存在着亮度和裁剪误差,提出基于不变矩景象匹配算法,并对矩不变量作如下改进：用原点矩代替中心矩,克服平移不变性给不变矩带来的影响;用圆形测量窗代替方形测量窗消除因旋转而产生的裁剪误差;引入亮度因子,克服基准图和匹配图之间因亮度差异带来的影响;采用金字塔算法减少搜索次数,提高匹配速度。实验表明,基于不变矩下视影像匹配算法具有较好稳定性,匹配准确率较高。